2023-10-16T08:00:17.315 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T08:00:17.316 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915 --name rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/orig.config.yaml 2023-10-16T08:00:17.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T08:00:17.494 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915 branch: rishabh-2023oct13 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: null first_in_suite: false job_id: '7428915' kernel: flavor: default kdb: true sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: rishabh-2023oct13 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023oct13-qa1 sha1: 75071e9ab3c3bca3308a5f42b10f82d632c58444 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6006 sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 sleep_before_teardown: 0 subset: 402/512 suite: fs suite_branch: rishabh-2023oct13-qa1 suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 75071e9ab3c3bca3308a5f42b10f82d632c58444 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqvVuFrg37eMep8ddVjkwbT5zCcErGvSYFtwEG/mXOCYAfK/iBsoIRy/HyXeG6IN0IuqYZf+79xZY0ZbxuRTuc= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8sPORMgXFVHvX5dfFzOhiDr4AyqKoQe4VOSoiyeAbGE2pFEXowpG0s50XRUwhLwjVPkXWoBj9ftaoMz8BBBHY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_06:54:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-16T08:00:17.495 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa; will attempt to use it 2023-10-16T08:00:17.495 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks 2023-10-16T08:00:17.496 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-16T08:00:17.498 INFO:teuthology.task.internal:Checking packages... 2023-10-16T08:00:17.519 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0' 2023-10-16T08:00:17.519 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-16T08:00:17.520 INFO:teuthology.packaging:ref: None 2023-10-16T08:00:17.520 INFO:teuthology.packaging:tag: None 2023-10-16T08:00:17.520 INFO:teuthology.packaging:branch: rishabh-2023oct13 2023-10-16T08:00:17.520 INFO:teuthology.packaging:sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T08:00:17.521 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=rishabh-2023oct13 2023-10-16T08:00:17.669 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:00:17.670 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-16T08:00:17.679 INFO:teuthology.task.internal:no buildpackages task found 2023-10-16T08:00:17.680 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-16T08:00:17.694 INFO:teuthology.task.internal:Saving configuration 2023-10-16T08:00:17.710 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-16T08:00:17.719 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T08:00:17.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 07:54:44.943355', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBhZzSCleKZjWeHkC48Af2Gc6KcG5H1wXGrUiX3TX1Eq'} 2023-10-16T08:00:17.781 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 07:54:44.945321', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIY8AZUKwI3GQIyJy/B7iKrJb4jdd4iK8QFvfXhfgBjj'} 2023-10-16T08:00:17.782 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-16T08:00:17.791 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-16T08:00:17.791 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-16T08:00:17.792 INFO:teuthology.run_tasks:Running task console_log... 2023-10-16T08:00:17.915 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd54b4a9a60>, signals=[15]) 2023-10-16T08:00:17.916 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-16T08:00:17.928 INFO:teuthology.task.internal:Opening connections... 2023-10-16T08:00:17.929 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:00:17.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:00:18.042 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:00:18.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:00:18.127 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-16T08:00:18.137 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-10-16T08:00:18.143 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-10-16T08:00:18.143 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-10-16T08:00:18.192 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-16T08:00:18.193 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2023-10-16T08:00:18.193 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="22.04" 2023-10-16T08:00:18.193 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-16T08:00:18.193 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=jammy 2023-10-16T08:00:18.194 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2023-10-16T08:00:18.194 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2023-10-16T08:00:18.194 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-16T08:00:18.194 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-16T08:00:18.195 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-16T08:00:18.195 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-16T08:00:18.195 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=jammy 2023-10-16T08:00:18.196 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-10-16T08:00:18.230 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-10-16T08:00:18.235 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-10-16T08:00:18.235 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-10-16T08:00:18.281 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-16T08:00:18.281 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Ubuntu" 2023-10-16T08:00:18.281 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="22.04" 2023-10-16T08:00:18.282 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-16T08:00:18.282 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_CODENAME=jammy 2023-10-16T08:00:18.282 INFO:teuthology.orchestra.run.smithi186.stdout:ID=ubuntu 2023-10-16T08:00:18.282 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE=debian 2023-10-16T08:00:18.282 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-16T08:00:18.283 INFO:teuthology.orchestra.run.smithi186.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-16T08:00:18.283 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-16T08:00:18.283 INFO:teuthology.orchestra.run.smithi186.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-16T08:00:18.283 INFO:teuthology.orchestra.run.smithi186.stdout:UBUNTU_CODENAME=jammy 2023-10-16T08:00:18.284 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-10-16T08:00:18.312 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-16T08:00:18.325 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-16T08:00:18.342 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-16T08:00:18.342 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-10-16T08:00:18.345 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-10-16T08:00:18.350 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-16T08:00:18.361 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-16T08:00:18.362 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-10-16T08:00:18.390 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-10-16T08:00:18.435 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-16T08:00:18.520 INFO:teuthology.run_tasks:Running task kernel... 2023-10-16T08:00:18.541 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'} 2023-10-16T08:00:18.542 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}}, timeout 300 2023-10-16T08:00:18.542 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T08:00:18.543 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T08:00:18.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:00:18.548 DEBUG:teuthology.task.kernel:sha1 for mon.b is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:00:18.548 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T08:00:18.549 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2023-10-16T08:00:18.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:00:18.550 DEBUG:teuthology.task.kernel:sha1 for mon.a is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:00:18.550 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T08:00:18.551 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-10-16T08:00:18.597 INFO:teuthology.orchestra.run.smithi186.stdout:5.15.0-56-generic 2023-10-16T08:00:18.598 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:00:18.598 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T08:00:18.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:00:18.639 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-56-generic 2023-10-16T08:00:18.639 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:00:18.640 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T08:00:18.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:00:18.802 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2023-10-16T08:00:18.802 DEBUG:teuthology.orchestra.run.smithi186:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-16T08:00:18.825 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2023-10-16T08:00:18.825 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-16T08:00:18.826 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-16T08:00:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-10-16T08:00:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-16T08:00:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-16T08:00:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-16T08:00:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2023-10-16T08:00:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:hint: git config --global init.defaultBranch 2023-10-16T08:00:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2023-10-16T08:00:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-16T08:00:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-16T08:00:18.969 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2023-10-16T08:00:18.969 INFO:teuthology.orchestra.run.smithi032.stderr:hint: git branch -m 2023-10-16T08:00:18.970 INFO:teuthology.orchestra.run.smithi032.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-16T08:00:18.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-16T08:00:18.972 INFO:teuthology.orchestra.run.smithi186.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-10-16T08:00:18.972 INFO:teuthology.orchestra.run.smithi186.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-10-16T08:00:18.972 INFO:teuthology.orchestra.run.smithi186.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-10-16T08:00:18.973 INFO:teuthology.orchestra.run.smithi186.stderr:hint: 2023-10-16T08:00:18.973 INFO:teuthology.orchestra.run.smithi186.stderr:hint: git config --global init.defaultBranch 2023-10-16T08:00:18.973 INFO:teuthology.orchestra.run.smithi186.stderr:hint: 2023-10-16T08:00:18.973 INFO:teuthology.orchestra.run.smithi186.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-10-16T08:00:18.973 INFO:teuthology.orchestra.run.smithi186.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-10-16T08:00:18.973 INFO:teuthology.orchestra.run.smithi186.stderr:hint: 2023-10-16T08:00:18.974 INFO:teuthology.orchestra.run.smithi186.stderr:hint: git branch -m 2023-10-16T08:00:18.974 INFO:teuthology.orchestra.run.smithi186.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-16T08:00:18.975 DEBUG:teuthology.orchestra.run.smithi186:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-16T08:00:19.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-16T08:00:19.040 DEBUG:teuthology.orchestra.run.smithi186:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-16T08:00:19.094 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-16T08:00:19.098 DEBUG:teuthology.orchestra.run.smithi186:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-16T08:01:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-16T08:01:10.893 INFO:teuthology.orchestra.run.smithi032.stderr: * [new branch] main -> origin/main 2023-10-16T08:01:10.924 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190312 -> 20190312 2023-10-16T08:01:10.924 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190416 -> 20190416 2023-10-16T08:01:10.924 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190514 -> 20190514 2023-10-16T08:01:10.925 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190618 -> 20190618 2023-10-16T08:01:10.925 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190717 -> 20190717 2023-10-16T08:01:10.926 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190815 -> 20190815 2023-10-16T08:01:10.926 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190923 -> 20190923 2023-10-16T08:01:10.927 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191022 -> 20191022 2023-10-16T08:01:10.927 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191215 -> 20191215 2023-10-16T08:01:10.927 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200122 -> 20200122 2023-10-16T08:01:10.927 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200316 -> 20200316 2023-10-16T08:01:10.927 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200421 -> 20200421 2023-10-16T08:01:10.928 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200519 -> 20200519 2023-10-16T08:01:10.928 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200619 -> 20200619 2023-10-16T08:01:10.931 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200721 -> 20200721 2023-10-16T08:01:10.931 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200817 -> 20200817 2023-10-16T08:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200918 -> 20200918 2023-10-16T08:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201022 -> 20201022 2023-10-16T08:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201118 -> 20201118 2023-10-16T08:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201218 -> 20201218 2023-10-16T08:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210208 -> 20210208 2023-10-16T08:01:10.932 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210315 -> 20210315 2023-10-16T08:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210511 -> 20210511 2023-10-16T08:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210716 -> 20210716 2023-10-16T08:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210818 -> 20210818 2023-10-16T08:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210919 -> 20210919 2023-10-16T08:01:10.933 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211027 -> 20211027 2023-10-16T08:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211216 -> 20211216 2023-10-16T08:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220209 -> 20220209 2023-10-16T08:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220310 -> 20220310 2023-10-16T08:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220411 -> 20220411 2023-10-16T08:01:10.934 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220509 -> 20220509 2023-10-16T08:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220610 -> 20220610 2023-10-16T08:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220708 -> 20220708 2023-10-16T08:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220815 -> 20220815 2023-10-16T08:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220913 -> 20220913 2023-10-16T08:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221012 -> 20221012 2023-10-16T08:01:10.935 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221109 -> 20221109 2023-10-16T08:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221214 -> 20221214 2023-10-16T08:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230117 -> 20230117 2023-10-16T08:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230210 -> 20230210 2023-10-16T08:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230310 -> 20230310 2023-10-16T08:01:10.936 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230404 -> 20230404 2023-10-16T08:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230515 -> 20230515 2023-10-16T08:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230625 -> 20230625 2023-10-16T08:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230804 -> 20230804 2023-10-16T08:01:10.937 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230919 -> 20230919 2023-10-16T08:01:12.158 INFO:teuthology.orchestra.run.smithi186.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-16T08:01:12.158 INFO:teuthology.orchestra.run.smithi186.stderr: * [new branch] main -> origin/main 2023-10-16T08:01:12.188 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190312 -> 20190312 2023-10-16T08:01:12.188 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190416 -> 20190416 2023-10-16T08:01:12.188 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190514 -> 20190514 2023-10-16T08:01:12.188 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190618 -> 20190618 2023-10-16T08:01:12.189 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190717 -> 20190717 2023-10-16T08:01:12.190 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190815 -> 20190815 2023-10-16T08:01:12.190 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20190923 -> 20190923 2023-10-16T08:01:12.190 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20191022 -> 20191022 2023-10-16T08:01:12.190 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20191215 -> 20191215 2023-10-16T08:01:12.191 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200122 -> 20200122 2023-10-16T08:01:12.191 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200316 -> 20200316 2023-10-16T08:01:12.191 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200421 -> 20200421 2023-10-16T08:01:12.191 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200519 -> 20200519 2023-10-16T08:01:12.195 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200619 -> 20200619 2023-10-16T08:01:12.195 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200721 -> 20200721 2023-10-16T08:01:12.195 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200817 -> 20200817 2023-10-16T08:01:12.195 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20200918 -> 20200918 2023-10-16T08:01:12.195 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20201022 -> 20201022 2023-10-16T08:01:12.196 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20201118 -> 20201118 2023-10-16T08:01:12.196 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20201218 -> 20201218 2023-10-16T08:01:12.196 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20210208 -> 20210208 2023-10-16T08:01:12.196 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20210315 -> 20210315 2023-10-16T08:01:12.196 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20210511 -> 20210511 2023-10-16T08:01:12.197 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20210716 -> 20210716 2023-10-16T08:01:12.197 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20210818 -> 20210818 2023-10-16T08:01:12.197 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20210919 -> 20210919 2023-10-16T08:01:12.197 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20211027 -> 20211027 2023-10-16T08:01:12.197 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20211216 -> 20211216 2023-10-16T08:01:12.198 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220209 -> 20220209 2023-10-16T08:01:12.198 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220310 -> 20220310 2023-10-16T08:01:12.198 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220411 -> 20220411 2023-10-16T08:01:12.198 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220509 -> 20220509 2023-10-16T08:01:12.198 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220610 -> 20220610 2023-10-16T08:01:12.199 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220708 -> 20220708 2023-10-16T08:01:12.199 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220815 -> 20220815 2023-10-16T08:01:12.199 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20220913 -> 20220913 2023-10-16T08:01:12.199 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20221012 -> 20221012 2023-10-16T08:01:12.199 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20221109 -> 20221109 2023-10-16T08:01:12.199 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20221214 -> 20221214 2023-10-16T08:01:12.200 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230117 -> 20230117 2023-10-16T08:01:12.200 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230210 -> 20230210 2023-10-16T08:01:12.200 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230310 -> 20230310 2023-10-16T08:01:12.200 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230404 -> 20230404 2023-10-16T08:01:12.200 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230515 -> 20230515 2023-10-16T08:01:12.201 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230625 -> 20230625 2023-10-16T08:01:12.201 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230804 -> 20230804 2023-10-16T08:01:12.201 INFO:teuthology.orchestra.run.smithi186.stderr: * [new tag] 20230919 -> 20230919 2023-10-16T08:01:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:Updating files: 48% (1542/3159) Updating files: 49% (1548/3159) Updating files: 50% (1580/3159) Updating files: 51% (1612/3159) Updating files: 52% (1643/3159) Updating files: 53% (1675/3159) Updating files: 54% (1706/3159) Updating files: 55% (1738/3159) Updating files: 56% (1770/3159) Updating files: 57% (1801/3159) Updating files: 58% (1833/3159) Updating files: 59% (1864/3159) Updating files: 60% (1896/3159) Updating files: 61% (1927/3159) Updating files: 62% (1959/3159) Updating files: 63% (1991/3159) Updating files: 64% (2022/3159) Updating files: 64% (2027/3159) Updating files: 65% (2054/3159) Updating files: 66% (2085/3159) Updating files: 67% (2117/3159) Updating files: 68% (2149/3159) Updating files: 69% (2180/3159) Updating files: 70% (2212/3159) Updating files: 71% (2243/3159) Updating files: 72% (2275/3159) Updating files: 73% (2307/3159) Updating files: 74% (2338/3159) Updating files: 75% (2370/3159) Updating files: 76% (2401/3159) Updating files: 77% (2433/3159) Updating files: 78% (2465/3159) Updating files: 79% (2496/3159) Updating files: 80% (2528/3159) Updating files: 81% (2559/3159) Updating files: 81% (2573/3159) Updating files: 82% (2591/3159) Updating files: 83% (2622/3159) Updating files: 84% (2654/3159) Updating files: 85% (2686/3159) Updating files: 86% (2717/3159) Updating files: 87% (2749/3159) Updating files: 88% (2780/3159) Updating files: 89% (2812/3159) Updating files: 90% (2844/3159) Updating files: 91% (2875/3159) Updating files: 92% (2907/3159) Updating files: 93% (2938/3159) Updating files: 94% (2970/3159) Updating files: 95% (3002/3159) Updating files: 96% (3033/3159) Updating files: 97% (3065/3159) Updating files: 98% (3096/3159) Updating files: 99% (3128/3159) Updating files: 100% (3159/3159) Updating files: 100% (3159/3159), done. 2023-10-16T08:01:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:HEAD is now at 1be48f85 Merge branch 'pr-24-1697222431' into 'main' 2023-10-16T08:01:14.856 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.a... 2023-10-16T08:01:14.857 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:01:15.630 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-16T08:01:15.631 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-16T08:01:15.634 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-10-16T08:01:15.879 INFO:teuthology.orchestra.run.smithi186.stderr:Updating files: 51% (1629/3159) Updating files: 52% (1643/3159) Updating files: 53% (1675/3159) Updating files: 54% (1706/3159) Updating files: 55% (1738/3159) Updating files: 56% (1770/3159) Updating files: 57% (1801/3159) Updating files: 58% (1833/3159) Updating files: 59% (1864/3159) Updating files: 60% (1896/3159) Updating files: 61% (1927/3159) Updating files: 62% (1959/3159) Updating files: 63% (1991/3159) Updating files: 64% (2022/3159) Updating files: 65% (2054/3159) Updating files: 66% (2085/3159) Updating files: 66% (2104/3159) Updating files: 67% (2117/3159) Updating files: 68% (2149/3159) Updating files: 69% (2180/3159) Updating files: 70% (2212/3159) Updating files: 71% (2243/3159) Updating files: 72% (2275/3159) Updating files: 73% (2307/3159) Updating files: 74% (2338/3159) Updating files: 75% (2370/3159) Updating files: 76% (2401/3159) Updating files: 77% (2433/3159) Updating files: 78% (2465/3159) Updating files: 79% (2496/3159) Updating files: 80% (2528/3159) Updating files: 81% (2559/3159) Updating files: 81% (2587/3159) Updating files: 82% (2591/3159) Updating files: 83% (2622/3159) Updating files: 84% (2654/3159) Updating files: 85% (2686/3159) Updating files: 86% (2717/3159) Updating files: 87% (2749/3159) Updating files: 88% (2780/3159) Updating files: 89% (2812/3159) Updating files: 90% (2844/3159) Updating files: 91% (2875/3159) Updating files: 92% (2907/3159) Updating files: 93% (2938/3159) Updating files: 94% (2970/3159) Updating files: 95% (3002/3159) Updating files: 96% (3033/3159) Updating files: 97% (3065/3159) Updating files: 98% (3096/3159) Updating files: 99% (3128/3159) Updating files: 100% (3159/3159) Updating files: 100% (3159/3159), done. 2023-10-16T08:01:15.884 INFO:teuthology.orchestra.run.smithi186.stdout:HEAD is now at 1be48f85 Merge branch 'pr-24-1697222431' into 'main' 2023-10-16T08:01:15.904 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.b... 2023-10-16T08:01:15.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T08:01:18.684 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-16T08:01:18.685 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-16T08:01:18.688 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi186.front.sepia.ceph.com to complete... 2023-10-16T08:01:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16 08:01:23 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38909314/38909314] -> "/tmp/linux-image.deb" [1] 2023-10-16T08:01:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2023-10-16 08:01:23-- 2023-10-16T08:01:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 6.9s 2023-10-16T08:01:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 37M in 3.7s (9.91 MB/s) 2023-10-16T08:01:22.603 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.a... 2023-10-16T08:01:22.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo dpkg -i /tmp/linux-image.deb 2023-10-16T08:01:22.862 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-16T08:01:24.196 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16 08:01:24 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38909314/38909314] -> "/tmp/linux-image.deb" [1] 2023-10-16T08:01:24.196 INFO:teuthology.orchestra.run.smithi186.stderr:FINISHED --2023-10-16 08:01:24-- 2023-10-16T08:01:24.196 INFO:teuthology.orchestra.run.smithi186.stderr:Total wall clock time: 5.5s 2023-10-16T08:01:24.196 INFO:teuthology.orchestra.run.smithi186.stderr:Downloaded: 1 files, 37M in 3.8s (9.88 MB/s) 2023-10-16T08:01:24.198 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on mon.b... 2023-10-16T08:01:24.199 DEBUG:teuthology.orchestra.run.smithi186:> sudo dpkg -i /tmp/linux-image.deb 2023-10-16T08:01:24.450 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-16T08:01:24.831 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-16T08:01:24.835 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-16T08:01:24.959 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T08:01:26.389 INFO:teuthology.orchestra.run.smithi186.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-10-16T08:01:26.393 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-16T08:01:26.498 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T08:01:28.133 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T08:01:28.237 INFO:teuthology.orchestra.run.smithi032.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T08:01:29.773 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-16T08:01:29.898 INFO:teuthology.orchestra.run.smithi186.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T08:01:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-10-16T08:01:48.533 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:01:48.556 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-10-16T08:01:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T08:01:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T08:01:49.142 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:01:49.150 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:01:49.218 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:01:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:01:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:01:49.369 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:01:49.369 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:01:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-10-16T08:01:49.608 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-deb -c /tmp/linux-image.deb 2023-10-16T08:01:49.616 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./ 2023-10-16T08:01:49.616 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./boot/ 2023-10-16T08:01:49.616 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:56 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T08:01:49.629 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:56 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-16T08:01:49.629 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:56 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T08:01:49.650 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/ 2023-10-16T08:01:49.650 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/ 2023-10-16T08:01:49.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postinst.d/ 2023-10-16T08:01:49.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postrm.d/ 2023-10-16T08:01:49.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/preinst.d/ 2023-10-16T08:01:49.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/prerm.d/ 2023-10-16T08:01:49.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/ 2023-10-16T08:01:49.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/ 2023-10-16T08:01:49.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-16T08:01:49.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-16T08:01:49.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-16T08:01:49.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-16T08:01:49.653 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-16T08:01:49.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-16T08:01:49.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T08:01:49.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T08:01:49.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T08:01:49.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T08:01:49.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T08:01:49.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T08:01:49.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T08:01:49.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T08:01:49.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T08:01:49.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T08:01:49.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T08:01:49.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T08:01:49.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T08:01:49.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T08:01:49.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T08:01:49.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T08:01:49.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T08:01:49.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T08:01:49.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T08:01:49.659 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-16T08:01:49.659 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-16T08:01:49.659 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-16T08:01:49.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T08:01:49.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-16T08:01:49.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-16T08:01:49.660 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-16T08:01:49.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T08:01:49.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T08:01:49.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T08:01:49.671 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-16T08:01:49.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-16T08:01:49.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-16T08:01:49.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-16T08:01:49.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T08:01:49.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T08:01:49.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T08:01:49.672 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-16T08:01:49.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-16T08:01:49.673 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-16T08:01:49.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T08:01:49.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T08:01:49.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T08:01:49.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T08:01:49.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T08:01:49.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T08:01:49.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T08:01:49.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T08:01:49.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T08:01:49.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T08:01:49.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T08:01:49.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T08:01:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T08:01:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T08:01:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T08:01:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T08:01:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T08:01:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T08:01:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-16T08:01:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T08:01:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-16T08:01:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T08:01:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T08:01:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T08:01:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T08:01:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T08:01:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T08:01:49.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T08:01:49.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T08:01:49.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-16T08:01:49.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T08:01:49.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T08:01:49.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T08:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T08:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T08:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T08:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T08:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T08:01:49.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T08:01:49.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T08:01:49.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T08:01:49.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T08:01:49.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T08:01:49.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T08:01:49.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T08:01:49.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T08:01:49.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T08:01:49.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-16T08:01:49.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T08:01:49.683 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-16T08:01:49.683 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-16T08:01:49.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-16T08:01:49.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T08:01:49.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T08:01:49.684 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-16T08:01:49.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-16T08:01:49.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T08:01:49.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T08:01:49.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T08:01:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T08:01:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T08:01:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-16T08:01:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T08:01:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-16T08:01:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T08:01:49.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-16T08:01:49.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T08:01:49.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T08:01:49.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T08:01:49.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T08:01:49.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T08:01:49.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T08:01:49.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T08:01:49.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T08:01:49.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T08:01:49.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T08:01:49.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T08:01:49.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T08:01:49.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T08:01:49.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T08:01:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T08:01:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T08:01:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T08:01:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T08:01:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-16T08:01:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T08:01:49.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T08:01:49.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T08:01:49.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T08:01:49.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T08:01:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T08:01:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T08:01:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T08:01:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T08:01:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T08:01:49.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T08:01:49.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T08:01:49.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T08:01:49.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-16T08:01:49.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T08:01:49.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-16T08:01:49.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T08:01:49.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T08:01:49.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T08:01:49.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T08:01:49.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T08:01:49.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T08:01:49.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T08:01:49.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T08:01:49.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T08:01:49.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T08:01:49.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T08:01:49.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T08:01:49.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T08:01:49.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T08:01:49.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T08:01:49.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T08:01:49.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T08:01:49.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T08:01:49.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T08:01:49.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T08:01:49.696 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-16T08:01:49.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T08:01:49.697 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-16T08:01:49.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T08:01:49.697 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-16T08:01:49.697 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-16T08:01:49.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T08:01:49.698 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-16T08:01:49.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T08:01:49.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T08:01:49.698 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-16T08:01:49.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T08:01:49.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T08:01:49.699 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-16T08:01:49.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T08:01:49.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T08:01:49.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T08:01:49.749 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-16T08:01:49.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T08:01:49.749 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-16T08:01:49.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T08:01:49.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-16T08:01:49.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-16T08:01:49.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T08:01:49.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-16T08:01:49.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T08:01:49.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T08:01:49.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T08:01:49.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T08:01:49.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T08:01:49.752 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-16T08:01:49.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T08:01:49.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T08:01:49.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T08:01:49.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T08:01:49.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T08:01:49.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T08:01:49.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T08:01:49.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-16T08:01:49.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T08:01:49.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-16T08:01:49.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T08:01:49.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T08:01:49.754 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-16T08:01:49.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T08:01:49.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-16T08:01:49.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-16T08:01:49.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-16T08:01:49.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T08:01:49.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T08:01:49.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-16T08:01:49.758 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-16T08:01:49.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-16T08:01:49.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-16T08:01:49.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-16T08:01:49.758 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-16T08:01:49.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-16T08:01:49.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-16T08:01:49.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-16T08:01:49.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-16T08:01:49.759 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-16T08:01:49.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T08:01:49.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T08:01:49.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T08:01:49.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-16T08:01:49.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-16T08:01:49.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T08:01:49.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T08:01:49.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-16T08:01:49.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-16T08:01:49.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-16T08:01:49.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T08:01:49.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-16T08:01:49.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T08:01:49.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T08:01:49.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T08:01:49.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-16T08:01:49.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T08:01:49.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-16T08:01:49.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T08:01:49.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-16T08:01:49.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-16T08:01:49.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T08:01:49.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-16T08:01:49.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T08:01:49.764 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-16T08:01:49.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T08:01:49.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T08:01:49.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T08:01:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T08:01:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T08:01:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T08:01:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T08:01:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T08:01:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T08:01:49.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T08:01:49.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T08:01:49.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T08:01:49.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T08:01:49.767 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-16T08:01:49.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-16T08:01:49.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-16T08:01:49.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-16T08:01:49.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-16T08:01:49.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-16T08:01:49.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-16T08:01:49.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-16T08:01:49.768 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-16T08:01:49.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-16T08:01:49.768 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-16T08:01:49.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-16T08:01:49.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T08:01:49.769 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-16T08:01:49.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-16T08:01:49.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-16T08:01:49.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-16T08:01:49.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-16T08:01:49.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-16T08:01:49.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-16T08:01:49.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-16T08:01:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-16T08:01:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T08:01:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-16T08:01:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-16T08:01:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-16T08:01:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-16T08:01:49.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-16T08:01:49.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-16T08:01:49.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-16T08:01:49.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-16T08:01:49.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-16T08:01:49.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-16T08:01:49.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-16T08:01:49.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-16T08:01:49.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-16T08:01:49.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-16T08:01:49.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-16T08:01:49.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T08:01:49.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-16T08:01:49.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-16T08:01:49.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-16T08:01:49.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-16T08:01:49.775 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-16T08:01:49.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-16T08:01:49.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-16T08:01:49.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T08:01:49.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T08:01:49.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T08:01:49.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-16T08:01:49.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-16T08:01:49.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-16T08:01:49.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-16T08:01:49.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-16T08:01:49.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T08:01:49.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T08:01:49.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T08:01:49.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T08:01:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-16T08:01:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T08:01:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-16T08:01:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T08:01:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T08:01:49.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T08:01:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T08:01:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T08:01:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T08:01:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T08:01:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-16T08:01:49.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T08:01:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T08:01:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T08:01:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T08:01:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T08:01:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-16T08:01:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T08:01:49.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T08:01:49.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-16T08:01:49.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-16T08:01:49.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-16T08:01:49.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-16T08:01:49.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T08:01:49.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-16T08:01:49.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T08:01:49.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T08:01:49.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T08:01:49.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T08:01:49.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T08:01:49.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T08:01:49.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T08:01:49.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T08:01:49.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T08:01:49.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T08:01:49.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T08:01:49.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T08:01:49.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T08:01:49.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T08:01:49.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T08:01:49.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T08:01:49.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T08:01:49.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T08:01:49.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T08:01:49.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T08:01:49.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T08:01:49.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T08:01:49.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T08:01:49.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T08:01:49.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T08:01:49.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T08:01:49.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T08:01:49.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T08:01:49.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T08:01:49.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T08:01:49.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T08:01:49.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T08:01:49.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T08:01:49.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T08:01:49.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T08:01:49.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T08:01:49.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T08:01:49.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T08:01:49.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T08:01:49.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-16T08:01:49.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T08:01:49.790 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-16T08:01:49.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-16T08:01:49.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-16T08:01:49.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-16T08:01:49.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T08:01:49.791 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-16T08:01:49.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T08:01:49.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T08:01:49.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T08:01:49.791 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-16T08:01:49.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T08:01:49.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T08:01:49.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T08:01:49.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T08:01:49.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T08:01:49.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T08:01:49.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T08:01:49.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T08:01:49.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T08:01:49.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T08:01:49.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T08:01:49.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T08:01:49.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T08:01:49.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T08:01:49.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T08:01:49.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T08:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T08:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T08:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T08:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T08:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T08:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T08:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T08:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T08:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T08:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-16T08:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-16T08:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T08:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T08:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T08:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T08:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T08:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-16T08:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T08:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T08:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T08:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T08:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T08:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T08:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T08:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T08:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T08:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T08:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-16T08:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T08:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T08:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T08:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T08:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T08:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T08:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T08:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T08:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-16T08:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T08:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T08:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T08:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T08:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T08:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T08:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T08:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T08:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T08:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T08:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T08:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T08:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T08:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T08:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T08:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T08:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T08:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T08:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T08:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T08:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T08:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T08:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T08:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T08:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T08:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T08:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T08:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T08:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T08:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T08:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T08:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T08:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-16T08:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T08:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-16T08:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T08:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T08:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T08:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T08:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T08:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T08:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T08:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-16T08:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T08:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T08:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T08:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T08:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T08:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T08:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T08:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T08:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T08:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T08:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T08:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T08:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T08:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T08:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T08:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T08:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T08:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T08:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T08:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T08:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T08:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T08:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T08:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T08:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T08:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T08:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T08:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T08:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T08:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T08:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T08:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T08:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T08:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T08:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T08:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T08:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T08:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T08:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T08:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T08:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T08:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-16T08:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-16T08:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-16T08:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-16T08:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T08:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T08:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-16T08:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-16T08:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-16T08:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-16T08:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T08:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T08:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T08:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-16T08:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T08:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-16T08:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T08:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T08:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T08:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T08:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-16T08:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-16T08:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T08:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T08:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T08:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-16T08:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T08:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T08:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-16T08:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T08:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T08:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T08:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-16T08:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T08:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T08:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-16T08:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-16T08:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T08:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-16T08:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T08:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T08:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T08:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T08:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-16T08:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T08:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T08:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T08:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-16T08:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-16T08:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-16T08:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-16T08:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-16T08:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-16T08:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T08:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T08:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T08:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T08:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T08:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T08:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T08:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-16T08:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-16T08:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T08:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-16T08:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T08:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-16T08:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T08:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-16T08:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T08:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-16T08:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-16T08:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-16T08:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-16T08:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-16T08:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-16T08:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T08:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-16T08:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T08:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-16T08:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T08:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-16T08:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T08:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-16T08:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T08:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-16T08:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-16T08:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-16T08:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-16T08:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T08:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T08:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T08:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T08:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-16T08:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-16T08:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-16T08:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-16T08:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-16T08:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T08:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-16T08:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-16T08:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-16T08:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-16T08:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-16T08:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-16T08:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-16T08:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-16T08:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-16T08:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-16T08:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-16T08:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-16T08:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-16T08:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-16T08:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-16T08:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-16T08:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-16T08:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-16T08:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-16T08:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-16T08:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-16T08:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T08:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T08:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T08:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T08:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-16T08:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-16T08:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T08:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T08:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T08:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T08:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-16T08:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T08:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-16T08:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T08:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T08:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-16T08:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T08:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-16T08:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-16T08:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-16T08:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T08:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-16T08:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T08:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-16T08:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-16T08:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T08:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-16T08:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-16T08:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-16T08:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-16T08:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T08:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-16T08:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-16T08:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-16T08:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T08:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-16T08:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-16T08:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-16T08:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T08:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-16T08:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-16T08:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-16T08:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-16T08:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-16T08:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-16T08:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-16T08:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T08:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-16T08:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-16T08:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-16T08:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-16T08:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T08:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T08:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T08:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T08:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T08:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T08:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T08:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T08:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-16T08:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-16T08:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T08:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T08:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-16T08:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T08:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T08:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T08:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T08:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-16T08:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T08:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T08:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-16T08:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-16T08:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T08:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T08:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T08:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-16T08:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T08:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T08:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-16T08:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T08:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T08:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T08:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-16T08:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T08:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T08:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T08:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-16T08:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T08:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T08:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T08:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T08:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T08:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-16T08:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-16T08:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T08:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T08:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T08:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T08:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T08:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T08:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T08:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-16T08:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-16T08:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-16T08:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-16T08:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-16T08:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-16T08:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-16T08:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-16T08:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-16T08:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-16T08:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-16T08:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-16T08:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-16T08:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-16T08:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T08:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T08:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-16T08:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-16T08:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-16T08:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-16T08:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-16T08:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-16T08:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-16T08:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-16T08:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-16T08:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-16T08:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T08:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-16T08:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-16T08:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-16T08:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T08:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T08:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T08:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-16T08:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-16T08:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-16T08:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-16T08:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-16T08:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T08:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T08:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T08:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T08:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-16T08:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-16T08:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-16T08:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-16T08:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T08:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-16T08:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-16T08:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-16T08:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-16T08:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-16T08:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T08:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T08:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T08:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T08:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T08:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-16T08:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-16T08:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-16T08:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T08:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-16T08:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-16T08:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-16T08:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T08:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T08:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-16T08:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T08:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T08:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-16T08:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T08:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T08:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T08:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-16T08:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-16T08:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-16T08:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T08:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T08:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T08:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T08:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-16T08:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T08:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-16T08:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-16T08:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T08:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-16T08:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-16T08:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T08:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T08:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T08:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-16T08:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T08:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T08:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T08:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-16T08:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-16T08:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T08:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T08:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T08:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T08:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-16T08:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T08:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T08:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T08:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T08:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-16T08:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T08:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-16T08:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T08:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-16T08:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T08:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-16T08:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T08:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-16T08:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T08:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T08:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T08:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-16T08:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-16T08:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T08:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-16T08:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T08:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-16T08:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T08:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-16T08:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T08:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T08:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-16T08:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T08:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T08:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-16T08:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T08:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T08:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-16T08:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T08:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T08:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-16T08:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-16T08:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T08:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-16T08:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-16T08:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T08:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-16T08:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T08:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-16T08:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T08:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-16T08:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T08:01:49.888 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-16T08:01:49.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T08:01:49.888 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-16T08:01:49.888 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-16T08:01:49.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T08:01:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-16T08:01:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-16T08:01:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T08:01:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T08:01:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T08:01:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T08:01:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T08:01:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T08:01:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-16T08:01:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T08:01:49.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T08:01:49.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T08:01:49.982 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-16T08:01:49.983 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-16T08:01:49.983 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-16T08:01:49.983 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T08:01:49.983 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T08:01:49.984 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T08:01:49.984 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-16T08:01:49.984 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-16T08:01:49.984 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-16T08:01:49.984 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T08:01:49.984 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-16T08:01:49.985 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T08:01:49.985 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-16T08:01:49.985 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T08:01:49.985 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-16T08:01:49.985 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T08:01:49.986 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-16T08:01:49.986 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T08:01:49.986 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-16T08:01:49.986 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T08:01:49.986 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-16T08:01:49.987 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T08:01:49.987 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-16T08:01:49.987 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T08:01:49.987 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-16T08:01:49.987 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T08:01:49.988 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-16T08:01:49.988 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T08:01:49.988 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T08:01:49.988 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-16T08:01:49.988 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T08:01:49.988 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T08:01:49.989 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T08:01:49.989 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-16T08:01:49.989 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-16T08:01:49.989 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T08:01:49.989 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T08:01:49.990 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-16T08:01:49.990 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-16T08:01:49.990 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T08:01:49.990 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-16T08:01:49.990 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-16T08:01:49.991 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-16T08:01:49.991 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-16T08:01:49.991 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T08:01:49.991 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-16T08:01:49.991 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-16T08:01:49.991 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T08:01:49.992 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-16T08:01:49.992 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T08:01:49.992 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T08:01:49.992 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-16T08:01:49.992 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T08:01:49.993 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-16T08:01:49.993 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T08:01:49.993 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-16T08:01:49.993 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T08:01:49.993 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T08:01:49.993 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-16T08:01:49.994 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-16T08:01:49.994 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T08:01:49.994 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T08:01:49.994 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-16T08:01:49.994 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T08:01:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-16T08:01:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T08:01:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-16T08:01:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T08:01:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T08:01:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T08:01:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T08:01:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-16T08:01:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-16T08:01:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-16T08:01:49.996 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-16T08:01:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T08:01:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-16T08:01:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T08:01:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-16T08:01:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T08:01:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T08:01:49.998 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-16T08:01:49.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T08:01:49.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T08:01:49.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T08:01:49.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T08:01:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-16T08:01:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-16T08:01:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-16T08:01:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T08:01:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-16T08:01:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T08:01:50.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-16T08:01:50.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T08:01:50.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T08:01:50.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T08:01:50.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T08:01:50.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-16T08:01:50.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T08:01:50.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-16T08:01:50.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T08:01:50.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-16T08:01:50.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T08:01:50.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T08:01:50.002 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-16T08:01:50.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T08:01:50.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T08:01:50.002 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-16T08:01:50.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T08:01:50.003 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-16T08:01:50.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-16T08:01:50.003 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-16T08:01:50.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-16T08:01:50.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T08:01:50.004 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-16T08:01:50.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T08:01:50.004 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-16T08:01:50.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T08:01:50.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T08:01:50.005 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-16T08:01:50.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T08:01:50.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T08:01:50.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T08:01:50.005 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-16T08:01:50.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T08:01:50.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T08:01:50.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-16T08:01:50.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T08:01:50.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T08:01:50.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T08:01:50.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T08:01:50.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T08:01:50.007 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-16T08:01:50.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T08:01:50.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T08:01:50.008 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-16T08:01:50.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T08:01:50.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T08:01:50.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T08:01:50.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T08:01:50.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T08:01:50.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T08:01:50.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T08:01:50.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T08:01:50.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T08:01:50.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T08:01:50.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T08:01:50.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T08:01:50.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T08:01:50.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T08:01:50.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T08:01:50.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T08:01:50.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T08:01:50.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T08:01:50.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T08:01:50.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-16T08:01:50.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-16T08:01:50.012 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-16T08:01:50.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T08:01:50.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T08:01:50.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T08:01:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T08:01:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T08:01:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T08:01:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T08:01:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T08:01:50.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-16T08:01:50.014 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-16T08:01:50.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T08:01:50.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T08:01:50.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T08:01:50.014 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-16T08:01:50.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T08:01:50.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T08:01:50.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T08:01:50.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T08:01:50.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T08:01:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T08:01:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T08:01:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-16T08:01:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T08:01:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-16T08:01:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T08:01:50.017 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-16T08:01:50.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T08:01:50.017 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-16T08:01:50.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T08:01:50.017 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-16T08:01:50.018 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-16T08:01:50.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T08:01:50.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T08:01:50.018 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-16T08:01:50.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T08:01:50.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T08:01:50.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T08:01:50.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T08:01:50.019 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-16T08:01:50.019 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-16T08:01:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T08:01:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-16T08:01:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-16T08:01:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-16T08:01:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T08:01:50.020 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-16T08:01:50.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T08:01:50.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-16T08:01:50.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-16T08:01:50.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-16T08:01:50.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T08:01:50.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T08:01:50.022 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-16T08:01:50.022 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-16T08:01:50.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-16T08:01:50.022 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-16T08:01:50.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-16T08:01:50.023 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-16T08:01:50.023 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-16T08:01:50.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T08:01:50.023 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-16T08:01:50.023 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-16T08:01:50.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-16T08:01:50.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-16T08:01:50.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-16T08:01:50.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-16T08:01:50.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-16T08:01:50.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-16T08:01:50.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-16T08:01:50.025 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-16T08:01:50.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T08:01:50.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T08:01:50.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T08:01:50.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T08:01:50.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T08:01:50.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T08:01:50.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T08:01:50.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T08:01:50.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T08:01:50.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T08:01:50.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T08:01:50.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T08:01:50.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-16T08:01:50.028 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-16T08:01:50.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T08:01:50.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-16T08:01:50.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T08:01:50.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T08:01:50.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T08:01:50.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T08:01:50.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T08:01:50.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T08:01:50.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T08:01:50.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T08:01:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T08:01:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T08:01:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T08:01:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-16T08:01:50.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-16T08:01:50.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-16T08:01:50.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-16T08:01:50.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-16T08:01:50.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-16T08:01:50.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-16T08:01:50.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-16T08:01:50.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-16T08:01:50.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-16T08:01:50.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-16T08:01:50.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-16T08:01:50.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-16T08:01:50.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-16T08:01:50.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-16T08:01:50.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-16T08:01:50.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-16T08:01:50.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-16T08:01:50.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-16T08:01:50.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-16T08:01:50.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-16T08:01:50.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-16T08:01:50.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-16T08:01:50.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-16T08:01:50.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-16T08:01:50.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-16T08:01:50.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-16T08:01:50.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-16T08:01:50.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-16T08:01:50.035 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-16T08:01:50.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T08:01:50.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T08:01:50.036 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-16T08:01:50.036 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-16T08:01:50.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T08:01:50.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T08:01:50.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T08:01:50.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T08:01:50.037 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-16T08:01:50.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T08:01:50.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T08:01:50.038 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-16T08:01:50.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-16T08:01:50.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-16T08:01:50.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T08:01:50.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-16T08:01:50.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-16T08:01:50.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-16T08:01:50.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T08:01:50.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T08:01:50.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T08:01:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T08:01:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T08:01:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T08:01:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T08:01:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T08:01:50.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T08:01:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T08:01:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T08:01:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T08:01:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T08:01:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-16T08:01:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T08:01:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T08:01:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T08:01:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-16T08:01:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T08:01:50.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-16T08:01:50.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-16T08:01:50.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-16T08:01:50.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-16T08:01:50.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-16T08:01:50.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T08:01:50.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-16T08:01:50.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T08:01:50.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-16T08:01:50.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T08:01:50.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T08:01:50.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T08:01:50.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T08:01:50.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-16T08:01:50.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T08:01:50.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T08:01:50.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T08:01:50.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T08:01:50.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T08:01:50.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-16T08:01:50.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T08:01:50.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-16T08:01:50.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-16T08:01:50.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-16T08:01:50.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-16T08:01:50.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T08:01:50.047 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-16T08:01:50.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T08:01:50.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T08:01:50.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T08:01:50.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T08:01:50.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T08:01:50.048 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-16T08:01:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T08:01:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T08:01:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-16T08:01:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T08:01:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T08:01:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-16T08:01:50.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-16T08:01:50.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T08:01:50.050 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-16T08:01:50.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T08:01:50.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T08:01:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-16T08:01:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T08:01:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-16T08:01:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T08:01:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-16T08:01:50.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T08:01:50.052 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-16T08:01:50.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T08:01:50.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T08:01:50.052 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-16T08:01:50.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T08:01:50.053 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-16T08:01:50.053 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-16T08:01:50.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T08:01:50.053 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-16T08:01:50.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T08:01:50.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T08:01:50.054 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T08:01:50.054 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T08:01:50.054 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-16T08:01:50.054 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T08:01:50.054 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T08:01:50.055 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-16T08:01:50.055 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T08:01:50.055 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T08:01:50.055 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-16T08:01:50.055 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-16T08:01:50.056 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T08:01:50.056 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T08:01:50.056 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T08:01:50.056 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T08:01:50.056 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-16T08:01:50.057 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T08:01:50.057 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T08:01:50.057 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T08:01:50.057 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-16T08:01:50.057 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T08:01:50.057 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T08:01:50.058 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T08:01:50.058 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-16T08:01:50.058 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T08:01:50.058 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T08:01:50.058 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T08:01:50.059 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-16T08:01:50.059 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T08:01:50.059 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-16T08:01:50.059 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T08:01:50.059 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-16T08:01:50.059 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T08:01:50.060 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T08:01:50.060 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T08:01:50.060 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T08:01:50.060 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-16T08:01:50.060 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T08:01:50.061 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-16T08:01:50.061 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T08:01:50.061 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T08:01:50.061 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-16T08:01:50.061 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-16T08:01:50.062 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-16T08:01:50.062 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-16T08:01:50.062 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-16T08:01:50.062 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-16T08:01:50.062 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T08:01:50.062 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T08:01:50.063 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-16T08:01:50.063 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T08:01:50.063 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-16T08:01:50.063 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T08:01:50.063 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T08:01:50.064 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-16T08:01:50.064 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T08:01:50.064 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-16T08:01:50.064 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T08:01:50.064 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T08:01:50.065 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T08:01:50.065 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T08:01:50.065 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T08:01:50.065 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T08:01:50.065 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T08:01:50.065 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T08:01:50.066 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T08:01:50.066 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T08:01:50.066 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-16T08:01:50.066 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T08:01:50.066 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T08:01:50.067 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T08:01:50.067 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T08:01:50.067 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-16T08:01:50.067 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-16T08:01:50.067 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T08:01:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-16T08:01:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-16T08:01:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T08:01:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-16T08:01:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T08:01:50.068 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T08:01:50.069 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T08:01:50.069 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T08:01:50.069 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T08:01:50.069 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T08:01:50.069 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-16T08:01:50.069 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T08:01:50.070 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-16T08:01:50.070 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-16T08:01:50.070 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-16T08:01:50.070 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T08:01:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T08:01:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T08:01:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T08:01:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T08:01:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T08:01:50.071 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T08:01:50.072 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T08:01:50.072 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T08:01:50.072 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T08:01:50.072 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T08:01:50.072 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T08:01:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-16T08:01:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T08:01:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T08:01:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T08:01:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-16T08:01:50.073 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-16T08:01:50.074 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T08:01:50.074 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-16T08:01:50.074 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-16T08:01:50.074 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T08:01:50.074 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T08:01:50.075 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T08:01:50.075 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-16T08:01:50.075 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-16T08:01:50.075 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-16T08:01:50.075 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T08:01:50.075 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-16T08:01:50.076 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-16T08:01:50.076 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T08:01:50.076 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-16T08:01:50.076 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-16T08:01:50.076 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T08:01:50.077 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-16T08:01:50.077 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-16T08:01:50.077 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-16T08:01:50.077 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-16T08:01:50.077 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T08:01:50.078 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-16T08:01:50.078 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T08:01:50.078 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-16T08:01:50.078 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-16T08:01:50.078 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T08:01:50.078 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T08:01:50.079 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T08:01:50.079 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-16T08:01:50.079 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-16T08:01:50.079 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-16T08:01:50.079 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T08:01:50.080 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-16T08:01:50.080 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T08:01:50.080 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-16T08:01:50.080 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-16T08:01:50.080 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-16T08:01:50.081 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-16T08:01:50.081 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-16T08:01:50.081 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-16T08:01:50.081 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-16T08:01:50.081 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-16T08:01:50.082 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-16T08:01:50.082 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T08:01:50.082 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T08:01:50.082 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T08:01:50.082 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T08:01:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-16T08:01:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-16T08:01:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-16T08:01:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-16T08:01:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-16T08:01:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-16T08:01:50.084 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-16T08:01:50.084 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-16T08:01:50.084 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-16T08:01:50.084 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-16T08:01:50.084 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-16T08:01:50.085 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-16T08:01:50.085 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-16T08:01:50.085 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-16T08:01:50.085 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-16T08:01:50.085 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-16T08:01:50.086 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-16T08:01:50.086 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-16T08:01:50.086 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-16T08:01:50.086 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-16T08:01:50.086 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-16T08:01:50.087 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-16T08:01:50.087 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-16T08:01:50.087 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-16T08:01:50.087 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-16T08:01:50.087 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-16T08:01:50.087 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-16T08:01:50.088 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-16T08:01:50.088 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-16T08:01:50.088 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-16T08:01:50.088 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-16T08:01:50.088 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-16T08:01:50.089 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-16T08:01:50.089 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-16T08:01:50.089 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-16T08:01:50.089 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-16T08:01:50.089 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-16T08:01:50.090 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-16T08:01:50.090 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-16T08:01:50.090 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-16T08:01:50.090 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-16T08:01:50.090 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-16T08:01:50.090 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-16T08:01:50.091 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-16T08:01:50.091 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-16T08:01:50.091 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-16T08:01:50.091 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-16T08:01:50.091 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-16T08:01:50.092 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-16T08:01:50.092 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-16T08:01:50.092 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-16T08:01:50.092 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-16T08:01:50.092 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-16T08:01:50.093 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-16T08:01:50.093 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-16T08:01:50.093 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-16T08:01:50.093 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-16T08:01:50.093 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-16T08:01:50.093 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-16T08:01:50.094 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-16T08:01:50.094 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-16T08:01:50.094 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-16T08:01:50.094 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-16T08:01:50.094 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T08:01:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-16T08:01:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-16T08:01:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-16T08:01:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-16T08:01:50.095 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T08:01:50.096 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T08:01:50.096 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-16T08:01:50.096 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T08:01:50.096 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T08:01:50.096 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-16T08:01:50.096 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-16T08:01:50.097 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-16T08:01:50.097 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T08:01:50.097 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T08:01:50.097 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T08:01:50.097 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T08:01:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T08:01:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T08:01:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T08:01:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T08:01:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-16T08:01:50.098 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-16T08:01:50.099 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T08:01:50.099 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T08:01:50.099 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T08:01:50.099 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T08:01:50.099 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T08:01:50.100 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-16T08:01:50.100 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T08:01:50.100 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T08:01:50.100 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-16T08:01:50.100 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-16T08:01:50.101 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-16T08:01:50.101 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-16T08:01:50.101 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-16T08:01:50.101 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T08:01:50.101 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-16T08:01:50.102 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-16T08:01:50.102 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-16T08:01:50.102 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-16T08:01:50.102 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T08:01:50.102 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T08:01:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T08:01:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T08:01:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T08:01:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T08:01:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T08:01:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T08:01:50.104 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T08:01:50.104 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-16T08:01:50.104 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T08:01:50.104 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T08:01:50.104 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T08:01:50.105 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T08:01:50.105 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T08:01:50.105 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T08:01:50.105 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T08:01:50.105 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T08:01:50.106 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T08:01:50.106 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T08:01:50.106 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T08:01:50.106 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T08:01:50.106 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T08:01:50.107 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-16T08:01:50.107 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T08:01:50.107 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T08:01:50.107 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-16T08:01:50.107 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T08:01:50.108 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T08:01:50.108 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T08:01:50.108 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T08:01:50.108 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T08:01:50.108 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T08:01:50.108 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T08:01:50.109 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T08:01:50.109 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T08:01:50.109 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T08:01:50.109 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T08:01:50.109 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T08:01:50.110 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T08:01:50.110 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T08:01:50.110 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T08:01:50.110 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T08:01:50.110 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T08:01:50.110 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T08:01:50.111 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-16T08:01:50.111 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T08:01:50.111 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T08:01:50.111 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-16T08:01:50.111 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T08:01:50.112 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-16T08:01:50.112 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T08:01:50.112 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T08:01:50.112 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T08:01:50.112 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T08:01:50.113 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T08:01:50.113 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T08:01:50.113 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T08:01:50.113 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T08:01:50.113 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T08:01:50.113 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T08:01:50.114 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T08:01:50.114 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T08:01:50.114 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T08:01:50.114 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T08:01:50.114 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T08:01:50.115 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-16T08:01:50.115 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T08:01:50.115 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T08:01:50.115 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T08:01:50.115 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-16T08:01:50.116 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-16T08:01:50.116 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T08:01:50.116 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T08:01:50.116 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T08:01:50.116 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T08:01:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-16T08:01:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T08:01:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T08:01:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T08:01:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T08:01:50.117 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-16T08:01:50.118 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-16T08:01:50.118 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-16T08:01:50.118 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-16T08:01:50.118 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-16T08:01:50.118 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-16T08:01:50.119 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-16T08:01:50.119 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T08:01:50.119 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-16T08:01:50.119 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-16T08:01:50.119 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-16T08:01:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-16T08:01:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-16T08:01:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-16T08:01:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T08:01:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-16T08:01:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-16T08:01:50.121 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-16T08:01:50.121 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-16T08:01:50.121 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-16T08:01:50.121 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-16T08:01:50.121 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T08:01:50.121 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-16T08:01:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-16T08:01:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-16T08:01:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T08:01:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T08:01:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-16T08:01:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-16T08:01:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-16T08:01:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T08:01:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T08:01:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T08:01:50.123 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T08:01:50.124 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T08:01:50.124 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T08:01:50.124 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T08:01:50.124 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T08:01:50.124 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T08:01:50.125 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T08:01:50.125 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T08:01:50.125 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T08:01:50.125 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-16T08:01:50.125 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-16T08:01:50.125 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-16T08:01:50.126 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T08:01:50.126 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T08:01:50.126 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-16T08:01:50.126 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-16T08:01:50.126 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-16T08:01:50.127 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-16T08:01:50.127 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-16T08:01:50.127 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-16T08:01:50.127 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T08:01:50.127 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T08:01:50.128 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T08:01:50.128 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T08:01:50.128 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T08:01:50.128 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T08:01:50.128 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T08:01:50.128 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T08:01:50.129 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T08:01:50.129 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T08:01:50.129 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-16T08:01:50.129 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T08:01:50.129 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-16T08:01:50.130 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-16T08:01:50.130 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T08:01:50.130 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-16T08:01:50.130 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-16T08:01:50.130 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T08:01:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T08:01:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-16T08:01:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-16T08:01:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-16T08:01:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T08:01:50.131 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-16T08:01:50.132 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T08:01:50.132 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-16T08:01:50.132 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T08:01:50.132 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T08:01:50.132 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-16T08:01:50.133 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T08:01:50.133 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T08:01:50.133 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T08:01:50.133 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T08:01:50.133 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-16T08:01:50.134 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T08:01:50.134 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-16T08:01:50.134 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-16T08:01:50.134 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T08:01:50.134 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-16T08:01:50.134 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T08:01:50.135 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T08:01:50.135 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T08:01:50.135 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-16T08:01:50.135 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T08:01:50.135 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T08:01:50.136 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-16T08:01:50.136 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-16T08:01:50.136 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T08:01:50.136 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-16T08:01:50.136 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-16T08:01:50.136 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T08:01:50.137 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-16T08:01:50.137 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-16T08:01:50.137 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-16T08:01:50.137 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T08:01:50.137 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-16T08:01:50.138 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-16T08:01:50.138 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-16T08:01:50.138 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-16T08:01:50.138 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-16T08:01:50.138 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T08:01:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-16T08:01:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-16T08:01:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T08:01:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-16T08:01:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T08:01:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-16T08:01:50.140 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T08:01:50.140 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-16T08:01:50.140 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T08:01:50.140 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-16T08:01:50.140 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T08:01:50.141 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-16T08:01:50.141 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-16T08:01:50.141 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-16T08:01:50.141 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T08:01:50.141 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-16T08:01:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T08:01:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-16T08:01:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-16T08:01:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-16T08:01:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T08:01:50.142 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-16T08:01:50.143 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T08:01:50.143 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-16T08:01:50.143 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-16T08:01:50.143 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-16T08:01:50.143 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T08:01:50.144 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-16T08:01:50.144 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T08:01:50.144 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-16T08:01:50.144 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T08:01:50.144 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-16T08:01:50.145 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T08:01:50.145 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-16T08:01:50.145 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T08:01:50.145 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-16T08:01:50.145 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-16T08:01:50.146 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-16T08:01:50.146 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T08:01:50.146 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-16T08:01:50.146 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T08:01:50.146 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-16T08:01:50.146 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T08:01:50.147 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-16T08:01:50.147 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T08:01:50.147 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-16T08:01:50.147 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T08:01:50.147 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-16T08:01:50.148 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-16T08:01:50.148 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T08:01:50.148 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-16T08:01:50.148 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T08:01:50.148 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-16T08:01:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T08:01:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T08:01:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-16T08:01:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T08:01:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T08:01:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-16T08:01:50.150 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T08:01:50.150 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T08:01:50.150 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-16T08:01:50.150 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T08:01:50.150 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-16T08:01:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T08:01:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-16T08:01:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T08:01:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T08:01:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T08:01:50.151 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T08:01:50.152 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T08:01:50.152 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T08:01:50.152 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T08:01:50.152 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T08:01:50.152 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T08:01:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T08:01:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T08:01:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-16T08:01:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T08:01:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T08:01:50.153 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T08:01:50.154 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T08:01:50.154 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T08:01:50.154 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T08:01:50.154 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T08:01:50.154 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T08:01:50.155 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T08:01:50.155 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T08:01:50.155 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T08:01:50.155 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T08:01:50.155 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T08:01:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T08:01:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T08:01:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T08:01:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T08:01:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T08:01:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T08:01:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T08:01:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T08:01:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T08:01:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T08:01:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T08:01:50.158 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T08:01:50.158 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T08:01:50.158 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T08:01:50.158 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T08:01:50.158 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T08:01:50.159 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T08:01:50.159 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T08:01:50.159 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T08:01:50.159 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T08:01:50.159 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T08:01:50.159 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T08:01:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T08:01:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T08:01:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T08:01:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T08:01:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-16T08:01:50.161 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-16T08:01:50.161 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-16T08:01:50.161 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-16T08:01:50.161 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T08:01:50.161 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-16T08:01:50.162 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T08:01:50.162 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-16T08:01:50.162 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T08:01:50.162 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T08:01:50.162 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T08:01:50.162 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T08:01:50.163 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T08:01:50.163 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-16T08:01:50.163 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-16T08:01:50.163 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-16T08:01:50.163 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T08:01:50.164 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-16T08:01:50.164 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T08:01:50.164 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-16T08:01:50.164 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-16T08:01:50.164 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-16T08:01:50.164 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-16T08:01:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-16T08:01:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-16T08:01:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-16T08:01:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-16T08:01:50.165 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-16T08:01:50.166 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T08:01:50.166 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-16T08:01:50.166 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T08:01:50.166 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-16T08:01:50.166 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-16T08:01:50.166 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T08:01:50.167 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-16T08:01:50.167 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T08:01:50.167 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T08:01:50.167 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-16T08:01:50.169 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T08:01:50.169 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-16T08:01:50.169 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T08:01:50.169 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-16T08:01:50.169 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T08:01:50.170 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-16T08:01:50.170 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T08:01:50.170 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-16T08:01:50.170 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T08:01:50.170 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-16T08:01:50.171 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-16T08:01:50.171 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-16T08:01:50.171 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-16T08:01:50.171 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-16T08:01:50.171 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T08:01:50.171 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-16T08:01:50.172 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T08:01:50.172 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-16T08:01:50.172 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T08:01:50.172 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-16T08:01:50.172 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T08:01:50.173 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T08:01:50.173 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-16T08:01:50.173 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T08:01:50.173 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T08:01:50.173 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-16T08:01:50.174 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T08:01:50.174 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T08:01:50.174 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-16T08:01:50.174 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T08:01:50.174 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-16T08:01:50.174 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-16T08:01:50.175 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-16T08:01:50.175 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-16T08:01:50.175 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-16T08:01:50.175 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T08:01:50.175 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-16T08:01:50.176 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-16T08:01:50.176 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T08:01:50.176 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-16T08:01:50.176 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-16T08:01:50.176 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-16T08:01:50.177 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-16T08:01:50.177 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-16T08:01:50.177 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-16T08:01:50.177 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T08:01:50.177 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T08:01:50.177 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T08:01:50.178 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-16T08:01:50.178 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-16T08:01:50.178 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-16T08:01:50.178 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T08:01:50.178 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-16T08:01:50.179 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-16T08:01:50.179 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T08:01:50.179 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T08:01:50.179 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T08:01:50.179 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T08:01:50.180 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T08:01:50.180 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T08:01:50.180 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T08:01:50.180 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T08:01:50.180 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-16T08:01:50.180 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T08:01:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T08:01:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T08:01:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T08:01:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T08:01:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-16T08:01:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T08:01:50.182 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-16T08:01:50.182 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T08:01:50.182 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T08:01:50.182 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T08:01:50.182 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T08:01:50.183 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-16T08:01:50.183 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T08:01:50.183 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-16T08:01:50.183 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T08:01:50.183 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T08:01:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-16T08:01:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T08:01:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T08:01:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T08:01:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T08:01:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T08:01:50.185 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T08:01:50.185 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T08:01:50.185 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T08:01:50.185 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T08:01:50.185 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T08:01:50.186 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T08:01:50.186 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T08:01:50.186 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T08:01:50.186 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T08:01:50.186 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T08:01:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T08:01:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T08:01:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T08:01:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T08:01:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T08:01:50.187 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T08:01:50.188 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T08:01:50.188 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T08:01:50.188 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-16T08:01:50.188 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T08:01:50.188 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-16T08:01:50.189 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T08:01:50.189 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-16T08:01:50.189 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T08:01:50.189 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T08:01:50.189 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T08:01:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T08:01:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-16T08:01:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T08:01:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T08:01:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T08:01:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T08:01:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T08:01:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-16T08:01:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-16T08:01:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-16T08:01:50.191 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-16T08:01:50.192 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T08:01:50.192 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T08:01:50.192 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T08:01:50.192 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-16T08:01:50.192 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T08:01:50.192 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T08:01:50.193 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T08:01:50.193 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T08:01:50.193 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T08:01:50.193 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T08:01:50.193 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T08:01:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T08:01:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T08:01:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T08:01:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-16T08:01:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T08:01:50.194 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T08:01:50.195 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T08:01:50.195 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-16T08:01:50.195 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T08:01:50.195 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-16T08:01:50.195 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T08:01:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T08:01:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T08:01:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-16T08:01:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T08:01:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T08:01:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T08:01:50.197 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T08:01:50.197 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T08:01:50.197 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T08:01:50.197 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T08:01:50.197 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T08:01:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T08:01:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-16T08:01:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T08:01:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T08:01:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T08:01:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T08:01:50.199 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T08:01:50.199 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T08:01:50.199 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T08:01:50.199 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T08:01:50.199 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T08:01:50.200 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T08:01:50.200 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T08:01:50.200 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T08:01:50.200 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T08:01:50.200 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T08:01:50.200 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T08:01:50.201 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T08:01:50.201 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T08:01:50.201 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T08:01:50.201 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T08:01:50.201 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T08:01:50.202 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T08:01:50.202 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T08:01:50.202 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-16T08:01:50.202 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T08:01:50.202 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T08:01:50.203 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T08:01:50.203 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-16T08:01:50.203 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T08:01:50.203 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T08:01:50.203 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T08:01:50.203 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T08:01:50.204 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-16T08:01:50.204 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-16T08:01:50.204 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T08:01:50.204 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-16T08:01:50.204 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T08:01:50.204 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T08:01:50.205 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T08:01:50.205 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T08:01:50.205 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T08:01:50.205 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T08:01:50.205 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T08:01:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T08:01:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T08:01:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T08:01:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T08:01:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T08:01:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T08:01:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T08:01:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T08:01:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T08:01:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T08:01:50.207 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T08:01:50.208 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T08:01:50.208 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T08:01:50.208 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T08:01:50.208 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T08:01:50.208 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T08:01:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T08:01:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T08:01:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T08:01:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-16T08:01:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T08:01:50.209 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-16T08:01:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T08:01:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T08:01:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T08:01:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T08:01:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T08:01:50.211 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T08:01:50.211 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T08:01:50.211 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-16T08:01:50.211 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-16T08:01:50.211 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-16T08:01:50.212 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T08:01:50.212 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-16T08:01:50.212 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-16T08:01:50.212 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T08:01:50.212 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-16T08:01:50.212 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T08:01:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-16T08:01:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-16T08:01:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T08:01:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T08:01:50.213 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T08:01:50.214 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T08:01:50.214 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T08:01:50.214 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T08:01:50.214 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T08:01:50.214 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T08:01:50.214 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T08:01:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T08:01:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T08:01:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T08:01:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T08:01:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T08:01:50.216 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T08:01:50.216 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T08:01:50.216 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-16T08:01:50.216 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T08:01:50.216 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T08:01:50.217 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T08:01:50.217 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T08:01:50.217 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T08:01:50.217 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T08:01:50.217 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T08:01:50.218 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T08:01:50.218 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T08:01:50.218 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T08:01:50.218 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T08:01:50.218 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T08:01:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T08:01:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T08:01:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T08:01:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T08:01:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T08:01:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T08:01:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T08:01:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T08:01:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T08:01:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T08:01:50.220 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T08:01:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T08:01:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T08:01:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T08:01:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T08:01:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T08:01:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T08:01:50.222 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T08:01:50.222 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T08:01:50.222 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T08:01:50.222 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T08:01:50.222 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T08:01:50.223 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T08:01:50.223 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T08:01:50.223 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T08:01:50.223 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T08:01:50.223 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T08:01:50.224 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T08:01:50.224 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T08:01:50.224 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T08:01:50.224 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-16T08:01:50.224 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T08:01:50.225 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T08:01:50.225 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T08:01:50.225 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T08:01:50.225 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T08:01:50.225 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T08:01:50.225 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T08:01:50.226 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T08:01:50.226 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T08:01:50.226 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T08:01:50.226 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T08:01:50.226 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T08:01:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T08:01:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T08:01:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T08:01:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-16T08:01:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T08:01:50.227 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T08:01:50.228 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T08:01:50.228 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T08:01:50.228 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T08:01:50.228 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-16T08:01:50.228 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-16T08:01:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-16T08:01:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T08:01:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T08:01:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T08:01:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T08:01:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T08:01:50.230 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T08:01:50.230 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T08:01:50.230 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T08:01:50.230 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T08:01:50.230 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T08:01:50.231 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T08:01:50.231 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T08:01:50.231 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T08:01:50.231 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T08:01:50.231 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T08:01:50.232 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T08:01:50.232 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T08:01:50.232 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T08:01:50.232 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T08:01:50.232 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T08:01:50.232 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T08:01:50.233 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T08:01:50.233 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T08:01:50.233 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T08:01:50.233 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T08:01:50.233 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T08:01:50.234 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T08:01:50.234 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T08:01:50.234 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T08:01:50.234 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T08:01:50.234 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T08:01:50.235 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T08:01:50.235 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T08:01:50.235 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T08:01:50.235 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T08:01:50.235 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T08:01:50.236 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T08:01:50.236 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T08:01:50.236 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T08:01:50.236 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T08:01:50.236 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T08:01:50.237 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T08:01:50.237 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T08:01:50.237 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T08:01:50.237 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T08:01:50.237 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T08:01:50.238 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T08:01:50.238 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T08:01:50.238 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T08:01:50.238 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T08:01:50.238 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T08:01:50.238 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T08:01:50.239 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T08:01:50.239 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T08:01:50.239 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T08:01:50.239 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T08:01:50.239 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T08:01:50.240 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T08:01:50.240 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T08:01:50.240 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T08:01:50.240 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T08:01:50.240 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T08:01:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T08:01:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T08:01:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T08:01:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T08:01:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T08:01:50.242 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T08:01:50.242 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T08:01:50.242 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-16T08:01:50.242 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T08:01:50.242 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T08:01:50.242 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-16T08:01:50.243 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T08:01:50.243 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-16T08:01:50.243 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T08:01:50.243 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-16T08:01:50.243 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T08:01:50.244 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T08:01:50.244 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T08:01:50.244 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T08:01:50.244 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-16T08:01:50.244 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T08:01:50.244 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-16T08:01:50.245 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-16T08:01:50.245 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-16T08:01:50.245 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-16T08:01:50.245 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T08:01:50.245 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T08:01:50.246 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T08:01:50.246 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-16T08:01:50.246 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-16T08:01:50.246 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-16T08:01:50.246 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T08:01:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T08:01:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T08:01:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T08:01:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T08:01:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T08:01:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T08:01:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T08:01:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T08:01:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T08:01:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T08:01:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T08:01:50.248 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-16T08:01:50.249 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T08:01:50.249 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T08:01:50.249 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T08:01:50.249 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T08:01:50.249 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T08:01:50.250 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T08:01:50.250 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T08:01:50.250 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T08:01:50.250 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T08:01:50.250 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T08:01:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T08:01:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T08:01:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T08:01:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T08:01:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-16T08:01:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T08:01:50.252 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T08:01:50.252 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T08:01:50.252 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T08:01:50.252 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T08:01:50.252 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T08:01:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T08:01:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T08:01:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T08:01:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T08:01:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T08:01:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T08:01:50.254 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T08:01:50.254 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T08:01:50.254 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T08:01:50.254 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T08:01:50.254 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T08:01:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T08:01:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-16T08:01:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T08:01:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T08:01:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-16T08:01:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-16T08:01:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T08:01:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T08:01:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T08:01:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-16T08:01:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T08:01:50.257 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-16T08:01:50.257 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T08:01:50.257 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T08:01:50.257 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-16T08:01:50.257 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T08:01:50.258 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-16T08:01:50.258 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T08:01:50.258 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T08:01:50.258 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T08:01:50.258 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T08:01:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T08:01:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-16T08:01:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-16T08:01:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-16T08:01:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-16T08:01:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T08:01:50.260 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-16T08:01:50.260 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-16T08:01:50.260 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-16T08:01:50.260 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T08:01:50.260 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T08:01:50.261 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T08:01:50.261 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T08:01:50.261 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-16T08:01:50.261 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T08:01:50.261 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T08:01:50.262 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T08:01:50.262 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T08:01:50.262 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 210 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T08:01:50.262 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T08:01:50.262 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 879 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T08:01:50.262 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T08:01:50.263 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T08:01:50.263 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/ 2023-10-16T08:01:50.263 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/ 2023-10-16T08:01:50.263 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/ 2023-10-16T08:01:50.263 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-16T08:01:50.264 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 175 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-16T08:01:50.264 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 691 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-16T08:01:50.265 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T08:01:50.265 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-16T08:01:50.266 DEBUG:teuthology.orchestra.run.smithi032:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-16T08:01:50.273 INFO:teuthology.orchestra.run.smithi032.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T08:01:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T08:01:50.274 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T08:01:50.275 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-16T08:01:50.275 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-16T08:01:50.275 DEBUG:teuthology.orchestra.run.smithi032:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-16T08:01:50.317 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-10-16T08:01:50.365 INFO:teuthology.orchestra.run.smithi186.stderr:Sourcing file `/etc/default/grub' 2023-10-16T08:01:50.387 INFO:teuthology.orchestra.run.smithi186.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:01:50.420 INFO:teuthology.orchestra.run.smithi186.stderr:Generating grub configuration file ... 2023-10-16T08:01:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-10-16T08:01:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:01:50.722 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T08:01:50.741 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T08:01:50.741 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-10-16T08:01:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T08:01:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T08:01:51.053 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:01:51.061 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:01:51.117 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:01:51.122 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:01:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:01:51.143 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:01:51.185 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:01:51.190 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:01:51.284 INFO:teuthology.orchestra.run.smithi186.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:01:51.285 INFO:teuthology.orchestra.run.smithi186.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:01:51.285 INFO:teuthology.orchestra.run.smithi186.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:01:51.316 INFO:teuthology.orchestra.run.smithi186.stderr:done 2023-10-16T08:01:51.331 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:01:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:01:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:01:51.353 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-10-16T08:01:51.505 DEBUG:teuthology.orchestra.run.smithi186:> dpkg-deb -c /tmp/linux-image.deb 2023-10-16T08:01:51.513 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./ 2023-10-16T08:01:51.513 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./boot/ 2023-10-16T08:01:51.513 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:56 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T08:01:51.523 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:56 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-16T08:01:51.523 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:56 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T08:01:51.540 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/ 2023-10-16T08:01:51.540 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/ 2023-10-16T08:01:51.540 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postinst.d/ 2023-10-16T08:01:51.541 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/postrm.d/ 2023-10-16T08:01:51.541 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/preinst.d/ 2023-10-16T08:01:51.541 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./etc/kernel/prerm.d/ 2023-10-16T08:01:51.541 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/ 2023-10-16T08:01:51.541 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/ 2023-10-16T08:01:51.542 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-16T08:01:51.542 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-16T08:01:51.542 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-16T08:01:51.542 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-16T08:01:51.542 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-16T08:01:51.543 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-16T08:01:51.545 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T08:01:51.545 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T08:01:51.546 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T08:01:51.546 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T08:01:51.546 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T08:01:51.546 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T08:01:51.546 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T08:01:51.547 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T08:01:51.547 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T08:01:51.547 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T08:01:51.547 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T08:01:51.547 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T08:01:51.547 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T08:01:51.548 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T08:01:51.548 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T08:01:51.548 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T08:01:51.548 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T08:01:51.548 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T08:01:51.549 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T08:01:51.549 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-16T08:01:51.549 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-16T08:01:51.549 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-16T08:01:51.549 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T08:01:51.550 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-16T08:01:51.550 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-16T08:01:51.550 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-16T08:01:51.550 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T08:01:51.550 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T08:01:51.551 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T08:01:51.594 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-16T08:01:51.594 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-16T08:01:51.594 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-16T08:01:51.595 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-16T08:01:51.595 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T08:01:51.595 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T08:01:51.595 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T08:01:51.595 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-16T08:01:51.596 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-16T08:01:51.596 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-16T08:01:51.596 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T08:01:51.596 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T08:01:51.596 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T08:01:51.597 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T08:01:51.597 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T08:01:51.597 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T08:01:51.597 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T08:01:51.597 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T08:01:51.597 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T08:01:51.598 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T08:01:51.598 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T08:01:51.598 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T08:01:51.598 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T08:01:51.598 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T08:01:51.599 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T08:01:51.599 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T08:01:51.599 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T08:01:51.599 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T08:01:51.599 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-16T08:01:51.600 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T08:01:51.600 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-16T08:01:51.600 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T08:01:51.600 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T08:01:51.600 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T08:01:51.600 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T08:01:51.601 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T08:01:51.601 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T08:01:51.601 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T08:01:51.601 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T08:01:51.601 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-16T08:01:51.602 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T08:01:51.602 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T08:01:51.602 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T08:01:51.602 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T08:01:51.602 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T08:01:51.603 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T08:01:51.603 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T08:01:51.603 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T08:01:51.603 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T08:01:51.603 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T08:01:51.603 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T08:01:51.604 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T08:01:51.604 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T08:01:51.604 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T08:01:51.604 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T08:01:51.604 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T08:01:51.604 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T08:01:51.605 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-16T08:01:51.605 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T08:01:51.605 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-16T08:01:51.605 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-16T08:01:51.605 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-16T08:01:51.606 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T08:01:51.606 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T08:01:51.606 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-16T08:01:51.606 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-16T08:01:51.606 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T08:01:51.607 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T08:01:51.607 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T08:01:51.607 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T08:01:51.607 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T08:01:51.607 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-16T08:01:51.607 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T08:01:51.609 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-16T08:01:51.609 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T08:01:51.609 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-16T08:01:51.610 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T08:01:51.610 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T08:01:51.610 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T08:01:51.610 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T08:01:51.610 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T08:01:51.611 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T08:01:51.611 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T08:01:51.611 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T08:01:51.611 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T08:01:51.611 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T08:01:51.612 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T08:01:51.612 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T08:01:51.612 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T08:01:51.612 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T08:01:51.612 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T08:01:51.612 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T08:01:51.613 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T08:01:51.613 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T08:01:51.613 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-16T08:01:51.613 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T08:01:51.613 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T08:01:51.614 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T08:01:51.614 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T08:01:51.614 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T08:01:51.614 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T08:01:51.614 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T08:01:51.615 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T08:01:51.615 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T08:01:51.615 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T08:01:51.615 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T08:01:51.615 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T08:01:51.616 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T08:01:51.616 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-16T08:01:51.616 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T08:01:51.616 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-16T08:01:51.616 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T08:01:51.616 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T08:01:51.617 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T08:01:51.617 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T08:01:51.617 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T08:01:51.617 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T08:01:51.617 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T08:01:51.618 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T08:01:51.618 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T08:01:51.618 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T08:01:51.618 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T08:01:51.618 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T08:01:51.619 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T08:01:51.619 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T08:01:51.619 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T08:01:51.619 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T08:01:51.619 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T08:01:51.619 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T08:01:51.620 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T08:01:51.620 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T08:01:51.620 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-16T08:01:51.620 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T08:01:51.620 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-16T08:01:51.621 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T08:01:51.621 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-16T08:01:51.621 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-16T08:01:51.621 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T08:01:51.621 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-16T08:01:51.621 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T08:01:51.622 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T08:01:51.622 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-16T08:01:51.622 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T08:01:51.622 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T08:01:51.622 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-16T08:01:51.623 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T08:01:51.623 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T08:01:51.623 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T08:01:51.623 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-16T08:01:51.623 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T08:01:51.624 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-16T08:01:51.624 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T08:01:51.624 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-16T08:01:51.624 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-16T08:01:51.624 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T08:01:51.625 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-16T08:01:51.625 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T08:01:51.625 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T08:01:51.625 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T08:01:51.625 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T08:01:51.625 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T08:01:51.626 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-16T08:01:51.626 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T08:01:51.626 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T08:01:51.626 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T08:01:51.626 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T08:01:51.627 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T08:01:51.627 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T08:01:51.627 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T08:01:51.627 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-16T08:01:51.627 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T08:01:51.628 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-16T08:01:51.628 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T08:01:51.628 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T08:01:51.628 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-16T08:01:51.628 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T08:01:51.628 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-16T08:01:51.629 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-16T08:01:51.629 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-16T08:01:51.629 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T08:01:51.629 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T08:01:51.629 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-16T08:01:51.629 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-16T08:01:51.630 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-16T08:01:51.630 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-16T08:01:51.630 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-16T08:01:51.630 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-16T08:01:51.630 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-16T08:01:51.631 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-16T08:01:51.631 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-16T08:01:51.631 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-16T08:01:51.631 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-16T08:01:51.631 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T08:01:51.632 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T08:01:51.632 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T08:01:51.632 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-16T08:01:51.632 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-16T08:01:51.632 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T08:01:51.632 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T08:01:51.633 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-16T08:01:51.633 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-16T08:01:51.633 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-16T08:01:51.633 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T08:01:51.633 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-16T08:01:51.634 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T08:01:51.634 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T08:01:51.634 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T08:01:51.634 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-16T08:01:51.634 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T08:01:51.635 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-16T08:01:51.635 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T08:01:51.635 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-16T08:01:51.635 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-16T08:01:51.635 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T08:01:51.636 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-16T08:01:51.636 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T08:01:51.636 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-16T08:01:51.636 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T08:01:51.636 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T08:01:51.636 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T08:01:51.637 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T08:01:51.637 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T08:01:51.637 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T08:01:51.637 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T08:01:51.637 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T08:01:51.638 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T08:01:51.638 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T08:01:51.638 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T08:01:51.638 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T08:01:51.638 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T08:01:51.638 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-16T08:01:51.639 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-16T08:01:51.639 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-16T08:01:51.639 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-16T08:01:51.639 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-16T08:01:51.639 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-16T08:01:51.640 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-16T08:01:51.640 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-16T08:01:51.640 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-16T08:01:51.640 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-16T08:01:51.640 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-16T08:01:51.641 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-16T08:01:51.641 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T08:01:51.641 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-16T08:01:51.641 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-16T08:01:51.641 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-16T08:01:51.641 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-16T08:01:51.642 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-16T08:01:51.642 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-16T08:01:51.642 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-16T08:01:51.642 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-16T08:01:51.642 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-16T08:01:51.643 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T08:01:51.643 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-16T08:01:51.643 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-16T08:01:51.643 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-16T08:01:51.643 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-16T08:01:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-16T08:01:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-16T08:01:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-16T08:01:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-16T08:01:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-16T08:01:51.644 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-16T08:01:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-16T08:01:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-16T08:01:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-16T08:01:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-16T08:01:51.645 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-16T08:01:51.646 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T08:01:51.646 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-16T08:01:51.646 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-16T08:01:51.646 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-16T08:01:51.646 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-16T08:01:51.646 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-16T08:01:51.647 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-16T08:01:51.647 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-16T08:01:51.647 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T08:01:51.647 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T08:01:51.647 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T08:01:51.648 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-16T08:01:51.648 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-16T08:01:51.648 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-16T08:01:51.648 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-16T08:01:51.648 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-16T08:01:51.649 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T08:01:51.649 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T08:01:51.649 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T08:01:51.649 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T08:01:51.649 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-16T08:01:51.649 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T08:01:51.650 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-16T08:01:51.650 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T08:01:51.650 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T08:01:51.650 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T08:01:51.650 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T08:01:51.651 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T08:01:51.651 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T08:01:51.651 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T08:01:51.651 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-16T08:01:51.651 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T08:01:51.652 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T08:01:51.652 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T08:01:51.652 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T08:01:51.652 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T08:01:51.652 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-16T08:01:51.652 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T08:01:51.653 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T08:01:51.653 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-16T08:01:51.653 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-16T08:01:51.653 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-16T08:01:51.653 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-16T08:01:51.653 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T08:01:51.654 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-16T08:01:51.654 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T08:01:51.654 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T08:01:51.654 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T08:01:51.654 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T08:01:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T08:01:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T08:01:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T08:01:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T08:01:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T08:01:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T08:01:51.656 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T08:01:51.656 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T08:01:51.656 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T08:01:51.656 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T08:01:51.656 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T08:01:51.656 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T08:01:51.657 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T08:01:51.657 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T08:01:51.657 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T08:01:51.657 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T08:01:51.657 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T08:01:51.658 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T08:01:51.658 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T08:01:51.658 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T08:01:51.658 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T08:01:51.658 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T08:01:51.658 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T08:01:51.659 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T08:01:51.659 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T08:01:51.659 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T08:01:51.659 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T08:01:51.659 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T08:01:51.660 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T08:01:51.660 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T08:01:51.660 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T08:01:51.660 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T08:01:51.660 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T08:01:51.660 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T08:01:51.661 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T08:01:51.661 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-16T08:01:51.661 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T08:01:51.661 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-16T08:01:51.661 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-16T08:01:51.662 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-16T08:01:51.662 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-16T08:01:51.662 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T08:01:51.662 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-16T08:01:51.662 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T08:01:51.663 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T08:01:51.663 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T08:01:51.663 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-16T08:01:51.663 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T08:01:51.663 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T08:01:51.663 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T08:01:51.664 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T08:01:51.664 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T08:01:51.664 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T08:01:51.664 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T08:01:51.664 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T08:01:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T08:01:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T08:01:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T08:01:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T08:01:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T08:01:51.665 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T08:01:51.666 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T08:01:51.666 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T08:01:51.666 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T08:01:51.666 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T08:01:51.666 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T08:01:51.666 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T08:01:51.667 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T08:01:51.667 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T08:01:51.667 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T08:01:51.667 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T08:01:51.667 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T08:01:51.668 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-16T08:01:51.668 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-16T08:01:51.668 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T08:01:51.668 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T08:01:51.668 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T08:01:51.668 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T08:01:51.669 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T08:01:51.669 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-16T08:01:51.669 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T08:01:51.669 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T08:01:51.669 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T08:01:51.669 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T08:01:51.670 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T08:01:51.670 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T08:01:51.670 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T08:01:51.670 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T08:01:51.670 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T08:01:51.671 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T08:01:51.671 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-16T08:01:51.671 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T08:01:51.671 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T08:01:51.671 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T08:01:51.671 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T08:01:51.672 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T08:01:51.672 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T08:01:51.672 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T08:01:51.672 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T08:01:51.672 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-16T08:01:51.673 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T08:01:51.673 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T08:01:51.673 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T08:01:51.673 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T08:01:51.673 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T08:01:51.673 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T08:01:51.674 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T08:01:51.674 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T08:01:51.674 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T08:01:51.674 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T08:01:51.674 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T08:01:51.675 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T08:01:51.675 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T08:01:51.675 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T08:01:51.675 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T08:01:51.675 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T08:01:51.675 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T08:01:51.676 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T08:01:51.676 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T08:01:51.676 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T08:01:51.676 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T08:01:51.676 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T08:01:51.676 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T08:01:51.677 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T08:01:51.677 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T08:01:51.677 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T08:01:51.677 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T08:01:51.677 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T08:01:51.678 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T08:01:51.678 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T08:01:51.678 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T08:01:51.678 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T08:01:51.678 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-16T08:01:51.678 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T08:01:51.679 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-16T08:01:51.679 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T08:01:51.679 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T08:01:51.679 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T08:01:51.679 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T08:01:51.680 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T08:01:51.680 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T08:01:51.680 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T08:01:51.680 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-16T08:01:51.680 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T08:01:51.680 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T08:01:51.681 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T08:01:51.681 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T08:01:51.681 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T08:01:51.681 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T08:01:51.681 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T08:01:51.681 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T08:01:51.682 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T08:01:51.682 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T08:01:51.682 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T08:01:51.682 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T08:01:51.682 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T08:01:51.683 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T08:01:51.683 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T08:01:51.683 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T08:01:51.683 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T08:01:51.683 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T08:01:51.683 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T08:01:51.684 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T08:01:51.684 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T08:01:51.684 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T08:01:51.684 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T08:01:51.684 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T08:01:51.685 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T08:01:51.685 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T08:01:51.685 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T08:01:51.685 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T08:01:51.685 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T08:01:51.685 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T08:01:51.686 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T08:01:51.686 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T08:01:51.686 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T08:01:51.686 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T08:01:51.686 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T08:01:51.687 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T08:01:51.687 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T08:01:51.687 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T08:01:51.687 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T08:01:51.687 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T08:01:51.687 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T08:01:51.688 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-16T08:01:51.688 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-16T08:01:51.688 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-16T08:01:51.688 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-16T08:01:51.688 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T08:01:51.689 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T08:01:51.689 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-16T08:01:51.689 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-16T08:01:51.689 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-16T08:01:51.689 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-16T08:01:51.689 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T08:01:51.690 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T08:01:51.690 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T08:01:51.690 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-16T08:01:51.690 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T08:01:51.690 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-16T08:01:51.691 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T08:01:51.691 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T08:01:51.691 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T08:01:51.691 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T08:01:51.691 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-16T08:01:51.691 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-16T08:01:51.692 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T08:01:51.692 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T08:01:51.692 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T08:01:51.692 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-16T08:01:51.692 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T08:01:51.693 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T08:01:51.693 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-16T08:01:51.693 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T08:01:51.693 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T08:01:51.693 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T08:01:51.693 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-16T08:01:51.694 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T08:01:51.694 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T08:01:51.694 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-16T08:01:51.694 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-16T08:01:51.694 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T08:01:51.695 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-16T08:01:51.695 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T08:01:51.695 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T08:01:51.695 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T08:01:51.695 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T08:01:51.695 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-16T08:01:51.696 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T08:01:51.696 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T08:01:51.696 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T08:01:51.696 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-16T08:01:51.696 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-16T08:01:51.697 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-16T08:01:51.697 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-16T08:01:51.697 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-16T08:01:51.697 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-16T08:01:51.697 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T08:01:51.698 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T08:01:51.698 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T08:01:51.698 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T08:01:51.698 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T08:01:51.698 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T08:01:51.698 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T08:01:51.699 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-16T08:01:51.699 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-16T08:01:51.699 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T08:01:51.699 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-16T08:01:51.699 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T08:01:51.699 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-16T08:01:51.700 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T08:01:51.700 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-16T08:01:51.700 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T08:01:51.700 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-16T08:01:51.700 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-16T08:01:51.701 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-16T08:01:51.701 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-16T08:01:51.701 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-16T08:01:51.701 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-16T08:01:51.701 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T08:01:51.701 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-16T08:01:51.702 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T08:01:51.702 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-16T08:01:51.702 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T08:01:51.702 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-16T08:01:51.702 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T08:01:51.703 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-16T08:01:51.703 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T08:01:51.703 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-16T08:01:51.703 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-16T08:01:51.703 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-16T08:01:51.704 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-16T08:01:51.704 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T08:01:51.704 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T08:01:51.704 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T08:01:51.704 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T08:01:51.704 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-16T08:01:51.705 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-16T08:01:51.705 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-16T08:01:51.705 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-16T08:01:51.705 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-16T08:01:51.705 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T08:01:51.706 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-16T08:01:51.706 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-16T08:01:51.706 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-16T08:01:51.706 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-16T08:01:51.706 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-16T08:01:51.707 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-16T08:01:51.707 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-16T08:01:51.707 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-16T08:01:51.707 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-16T08:01:51.707 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-16T08:01:51.707 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-16T08:01:51.708 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-16T08:01:51.708 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-16T08:01:51.708 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-16T08:01:51.708 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-16T08:01:51.708 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-16T08:01:51.709 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-16T08:01:51.709 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-16T08:01:51.709 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-16T08:01:51.709 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-16T08:01:51.709 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-16T08:01:51.710 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T08:01:51.710 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T08:01:51.710 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T08:01:51.710 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T08:01:51.710 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-16T08:01:51.710 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-16T08:01:51.711 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T08:01:51.711 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T08:01:51.711 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T08:01:51.711 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T08:01:51.711 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-16T08:01:51.712 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T08:01:51.712 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-16T08:01:51.712 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T08:01:51.712 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T08:01:51.712 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-16T08:01:51.712 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T08:01:51.713 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-16T08:01:51.713 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-16T08:01:51.713 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-16T08:01:51.713 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T08:01:51.713 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-16T08:01:51.714 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T08:01:51.714 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-16T08:01:51.714 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-16T08:01:51.714 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T08:01:51.714 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-16T08:01:51.714 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-16T08:01:51.715 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-16T08:01:51.715 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-16T08:01:51.715 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T08:01:51.715 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-16T08:01:51.715 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-16T08:01:51.716 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-16T08:01:51.716 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T08:01:51.716 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-16T08:01:51.716 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-16T08:01:51.716 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-16T08:01:51.716 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T08:01:51.717 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-16T08:01:51.717 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-16T08:01:51.717 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-16T08:01:51.717 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-16T08:01:51.717 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-16T08:01:51.718 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-16T08:01:51.718 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-16T08:01:51.718 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T08:01:51.718 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-16T08:01:51.718 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-16T08:01:51.718 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-16T08:01:51.719 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-16T08:01:51.719 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T08:01:51.719 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T08:01:51.719 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T08:01:51.719 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T08:01:51.720 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T08:01:51.720 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T08:01:51.720 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T08:01:51.720 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T08:01:51.720 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-16T08:01:51.720 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-16T08:01:51.721 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T08:01:51.721 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T08:01:51.721 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-16T08:01:51.721 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T08:01:51.721 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T08:01:51.722 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T08:01:51.722 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T08:01:51.722 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-16T08:01:51.722 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T08:01:51.722 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T08:01:51.722 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-16T08:01:51.723 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-16T08:01:51.723 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T08:01:51.723 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T08:01:51.723 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T08:01:51.723 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-16T08:01:51.724 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T08:01:51.724 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T08:01:51.724 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-16T08:01:51.724 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T08:01:51.724 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T08:01:51.724 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T08:01:51.725 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-16T08:01:51.725 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T08:01:51.725 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T08:01:51.725 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T08:01:51.725 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-16T08:01:51.726 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T08:01:51.726 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T08:01:51.726 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T08:01:51.726 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T08:01:51.726 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T08:01:51.727 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-16T08:01:51.727 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-16T08:01:51.727 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T08:01:51.727 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T08:01:51.727 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T08:01:51.727 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T08:01:51.728 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T08:01:51.728 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T08:01:51.848 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T08:01:51.848 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-16T08:01:51.848 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-16T08:01:51.848 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-16T08:01:51.848 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-16T08:01:51.849 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-16T08:01:51.849 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-16T08:01:51.849 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-16T08:01:51.849 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-16T08:01:51.849 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-16T08:01:51.850 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-16T08:01:51.850 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-16T08:01:51.850 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-16T08:01:51.850 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-16T08:01:51.850 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-16T08:01:51.851 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T08:01:51.851 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T08:01:51.851 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-16T08:01:51.851 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-16T08:01:51.851 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-16T08:01:51.851 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-16T08:01:51.852 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-16T08:01:51.852 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-16T08:01:51.852 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-16T08:01:51.852 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-16T08:01:51.852 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-16T08:01:51.853 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-16T08:01:51.853 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T08:01:51.853 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-16T08:01:51.853 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-16T08:01:51.853 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-16T08:01:51.853 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T08:01:51.854 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T08:01:51.854 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T08:01:51.854 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-16T08:01:51.854 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-16T08:01:51.854 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-16T08:01:51.855 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-16T08:01:51.855 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-16T08:01:51.855 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T08:01:51.855 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T08:01:51.855 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T08:01:51.856 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T08:01:51.856 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-16T08:01:51.856 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-16T08:01:51.856 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-16T08:01:51.856 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-16T08:01:51.856 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T08:01:51.857 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-16T08:01:51.857 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-16T08:01:51.857 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-16T08:01:51.857 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-16T08:01:51.857 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-16T08:01:51.858 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T08:01:51.858 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T08:01:51.858 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T08:01:51.858 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T08:01:51.858 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T08:01:51.859 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-16T08:01:51.859 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-16T08:01:51.859 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-16T08:01:51.859 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T08:01:51.859 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-16T08:01:51.859 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-16T08:01:51.860 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-16T08:01:51.860 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T08:01:51.860 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T08:01:51.860 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-16T08:01:51.860 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T08:01:51.861 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T08:01:51.861 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-16T08:01:51.861 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T08:01:51.861 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T08:01:51.861 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T08:01:51.861 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-16T08:01:51.862 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-16T08:01:51.862 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-16T08:01:51.862 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T08:01:51.862 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T08:01:51.862 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T08:01:51.863 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T08:01:51.863 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-16T08:01:51.863 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T08:01:51.863 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-16T08:01:51.863 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-16T08:01:51.864 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T08:01:51.864 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-16T08:01:51.864 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-16T08:01:51.864 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T08:01:51.864 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T08:01:51.864 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T08:01:51.865 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-16T08:01:51.865 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T08:01:51.865 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T08:01:51.865 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T08:01:51.865 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-16T08:01:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-16T08:01:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T08:01:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T08:01:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T08:01:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T08:01:51.866 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-16T08:01:51.867 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T08:01:51.867 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T08:01:51.867 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T08:01:51.867 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T08:01:51.867 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-16T08:01:51.868 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T08:01:51.868 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-16T08:01:51.868 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T08:01:51.868 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-16T08:01:51.868 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T08:01:51.868 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-16T08:01:51.869 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T08:01:51.869 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-16T08:01:51.869 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T08:01:51.869 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T08:01:51.869 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T08:01:51.870 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-16T08:01:51.870 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-16T08:01:51.870 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T08:01:51.870 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-16T08:01:51.870 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T08:01:51.870 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-16T08:01:51.871 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T08:01:51.871 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-16T08:01:51.871 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T08:01:51.871 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T08:01:51.871 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-16T08:01:51.872 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T08:01:51.872 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T08:01:51.872 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-16T08:01:51.872 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T08:01:51.872 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T08:01:51.873 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-16T08:01:51.873 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T08:01:51.873 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T08:01:51.873 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-16T08:01:51.873 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-16T08:01:51.873 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T08:01:51.874 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-16T08:01:51.874 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-16T08:01:51.874 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T08:01:51.874 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-16T08:01:51.874 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T08:01:51.875 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-16T08:01:51.875 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T08:01:51.875 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-16T08:01:51.875 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T08:01:51.875 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-16T08:01:51.875 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T08:01:51.876 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-16T08:01:51.876 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-16T08:01:51.876 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T08:01:51.876 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-16T08:01:51.876 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-16T08:01:51.877 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T08:01:51.877 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T08:01:51.877 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T08:01:51.877 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T08:01:51.877 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T08:01:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T08:01:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-16T08:01:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T08:01:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T08:01:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T08:01:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-16T08:01:51.879 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-16T08:01:51.879 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-16T08:01:51.879 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T08:01:51.879 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T08:01:51.879 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T08:01:51.880 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-16T08:01:51.880 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-16T08:01:51.880 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-16T08:01:51.880 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T08:01:51.880 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-16T08:01:51.880 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T08:01:51.881 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-16T08:01:51.881 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T08:01:51.881 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-16T08:01:51.881 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T08:01:51.881 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-16T08:01:51.882 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T08:01:51.882 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-16T08:01:51.882 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T08:01:51.882 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-16T08:01:51.882 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T08:01:51.883 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-16T08:01:51.883 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T08:01:51.883 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-16T08:01:51.883 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T08:01:51.883 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-16T08:01:51.883 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T08:01:51.884 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T08:01:51.884 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-16T08:01:51.884 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T08:01:51.884 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T08:01:51.884 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T08:01:51.885 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-16T08:01:51.885 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-16T08:01:51.885 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T08:01:51.885 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T08:01:51.885 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-16T08:01:51.886 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-16T08:01:51.886 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T08:01:51.886 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-16T08:01:51.886 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-16T08:01:51.886 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-16T08:01:51.886 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-16T08:01:51.887 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T08:01:51.887 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-16T08:01:51.887 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-16T08:01:51.887 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T08:01:51.887 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-16T08:01:51.888 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T08:01:51.888 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T08:01:51.888 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-16T08:01:51.888 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T08:01:51.888 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-16T08:01:51.889 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T08:01:51.889 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-16T08:01:51.889 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T08:01:51.889 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T08:01:51.889 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-16T08:01:51.889 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-16T08:01:51.890 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T08:01:51.890 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T08:01:51.890 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-16T08:01:51.890 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T08:01:51.890 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-16T08:01:51.891 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T08:01:51.891 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-16T08:01:51.891 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T08:01:51.891 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T08:01:51.891 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T08:01:51.891 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T08:01:51.892 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-16T08:01:51.892 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-16T08:01:51.892 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-16T08:01:51.892 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-16T08:01:51.892 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T08:01:51.893 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-16T08:01:51.893 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T08:01:51.893 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-16T08:01:51.893 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T08:01:51.893 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T08:01:51.893 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-16T08:01:51.894 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T08:01:51.894 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T08:01:51.894 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T08:01:51.894 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T08:01:51.894 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-16T08:01:51.895 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-16T08:01:51.895 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-16T08:01:51.895 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T08:01:51.895 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-16T08:01:51.895 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T08:01:51.896 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-16T08:01:51.896 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T08:01:51.896 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T08:01:51.896 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T08:01:51.896 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T08:01:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-16T08:01:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T08:01:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-16T08:01:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T08:01:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-16T08:01:51.897 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T08:01:51.898 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T08:01:51.898 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-16T08:01:51.898 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T08:01:51.898 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T08:01:51.898 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-16T08:01:51.899 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T08:01:51.899 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-16T08:01:51.899 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-16T08:01:51.899 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-16T08:01:51.899 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-16T08:01:51.900 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T08:01:51.900 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-16T08:01:51.900 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T08:01:51.900 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-16T08:01:51.900 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T08:01:51.900 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T08:01:51.901 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-16T08:01:51.901 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T08:01:51.901 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T08:01:51.901 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T08:01:51.901 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-16T08:01:51.902 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T08:01:51.902 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T08:01:51.902 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-16T08:01:51.902 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T08:01:51.902 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T08:01:51.902 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T08:01:51.903 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T08:01:51.903 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T08:01:51.903 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-16T08:01:51.903 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T08:01:51.903 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T08:01:51.904 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-16T08:01:51.904 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T08:01:51.904 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T08:01:51.904 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T08:01:51.904 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T08:01:51.905 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T08:01:51.905 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T08:01:51.905 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T08:01:51.905 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T08:01:51.905 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T08:01:51.905 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T08:01:51.906 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T08:01:51.906 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T08:01:51.906 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T08:01:51.906 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T08:01:51.906 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T08:01:51.907 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T08:01:51.907 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T08:01:51.907 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T08:01:51.907 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T08:01:51.907 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-16T08:01:51.908 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-16T08:01:51.908 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-16T08:01:51.908 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T08:01:51.908 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T08:01:51.908 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T08:01:51.908 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T08:01:51.909 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T08:01:51.909 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T08:01:51.909 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T08:01:51.909 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T08:01:51.909 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-16T08:01:51.910 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-16T08:01:51.910 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T08:01:51.910 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T08:01:51.910 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T08:01:51.910 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-16T08:01:51.911 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T08:01:51.911 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T08:01:51.911 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T08:01:51.911 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T08:01:51.911 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T08:01:51.911 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T08:01:51.912 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T08:01:51.912 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-16T08:01:51.912 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T08:01:51.912 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-16T08:01:51.912 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T08:01:51.913 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-16T08:01:51.913 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T08:01:51.913 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-16T08:01:51.913 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T08:01:51.913 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-16T08:01:51.913 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-16T08:01:51.914 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T08:01:51.914 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T08:01:51.914 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-16T08:01:51.914 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T08:01:51.914 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T08:01:51.915 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T08:01:51.915 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T08:01:51.915 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-16T08:01:51.915 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-16T08:01:51.915 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T08:01:51.915 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-16T08:01:51.916 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-16T08:01:51.916 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-16T08:01:51.916 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T08:01:51.916 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-16T08:01:51.916 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T08:01:51.917 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-16T08:01:51.917 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-16T08:01:51.917 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-16T08:01:51.917 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T08:01:51.917 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T08:01:51.917 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-16T08:01:51.918 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-16T08:01:51.918 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-16T08:01:51.918 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-16T08:01:51.918 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-16T08:01:51.918 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-16T08:01:51.919 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-16T08:01:51.919 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T08:01:51.919 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-16T08:01:51.919 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-16T08:01:51.919 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-16T08:01:51.919 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-16T08:01:51.920 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-16T08:01:51.920 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-16T08:01:51.920 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-16T08:01:51.920 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-16T08:01:51.920 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-16T08:01:51.921 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-16T08:01:51.921 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T08:01:51.921 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T08:01:51.921 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T08:01:51.921 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T08:01:51.922 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T08:01:51.922 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T08:01:51.922 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T08:01:51.922 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T08:01:51.922 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T08:01:51.923 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T08:01:51.923 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T08:01:51.923 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T08:01:51.923 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-16T08:01:51.923 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-16T08:01:51.923 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T08:01:51.924 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-16T08:01:51.924 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T08:01:51.924 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T08:01:51.924 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T08:01:51.924 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T08:01:51.925 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T08:01:51.925 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T08:01:51.925 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T08:01:51.925 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T08:01:51.925 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T08:01:51.925 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T08:01:51.926 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T08:01:51.926 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-16T08:01:51.926 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-16T08:01:51.926 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-16T08:01:51.926 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-16T08:01:51.927 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-16T08:01:51.927 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-16T08:01:51.927 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-16T08:01:51.927 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-16T08:01:51.927 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-16T08:01:51.927 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-16T08:01:51.928 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-16T08:01:51.928 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-16T08:01:51.928 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-16T08:01:51.928 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-16T08:01:51.928 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-16T08:01:51.929 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-16T08:01:51.929 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-16T08:01:51.929 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-16T08:01:51.929 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-16T08:01:51.929 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-16T08:01:51.929 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-16T08:01:51.930 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-16T08:01:51.930 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-16T08:01:51.930 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-16T08:01:51.930 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-16T08:01:51.930 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-16T08:01:51.931 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-16T08:01:51.931 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-16T08:01:51.931 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-16T08:01:51.931 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-16T08:01:51.931 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T08:01:51.931 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T08:01:51.932 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-16T08:01:51.932 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-16T08:01:51.932 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T08:01:51.932 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T08:01:51.932 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T08:01:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T08:01:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-16T08:01:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T08:01:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T08:01:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-16T08:01:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-16T08:01:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-16T08:01:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T08:01:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-16T08:01:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-16T08:01:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-16T08:01:51.935 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T08:01:51.935 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T08:01:51.935 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T08:01:51.935 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T08:01:51.935 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T08:01:51.936 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T08:01:51.936 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T08:01:51.936 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T08:01:51.936 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T08:01:51.936 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T08:01:51.936 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T08:01:51.937 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T08:01:51.937 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T08:01:51.937 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-16T08:01:51.937 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T08:01:51.937 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T08:01:51.938 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T08:01:51.938 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-16T08:01:51.938 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T08:01:51.938 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-16T08:01:51.938 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-16T08:01:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-16T08:01:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-16T08:01:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-16T08:01:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T08:01:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-16T08:01:51.939 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T08:01:51.940 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-16T08:01:51.940 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T08:01:51.940 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T08:01:51.940 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T08:01:51.940 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T08:01:51.941 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-16T08:01:51.941 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T08:01:51.941 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T08:01:51.941 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T08:01:51.941 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T08:01:51.941 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T08:01:51.942 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-16T08:01:51.942 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T08:01:51.942 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-16T08:01:51.942 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-16T08:01:51.942 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-16T08:01:51.943 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-16T08:01:51.943 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T08:01:51.943 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-16T08:01:51.943 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T08:01:51.943 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T08:01:51.944 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T08:01:51.944 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T08:01:51.944 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T08:01:51.944 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-16T08:01:51.944 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T08:01:51.945 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T08:01:51.945 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-16T08:01:51.945 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T08:01:51.945 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T08:01:51.945 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-16T08:01:51.945 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-16T08:01:51.946 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T08:01:51.946 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-16T08:01:51.946 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T08:01:51.946 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T08:01:51.946 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-16T08:01:51.947 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T08:01:51.947 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-16T08:01:51.947 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T08:01:51.947 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-16T08:01:51.947 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T08:01:51.948 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-16T08:01:51.948 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T08:01:51.948 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T08:01:51.948 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-16T08:01:51.948 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T08:01:51.948 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-16T08:01:51.949 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-16T08:01:51.949 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T08:01:51.949 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-16T08:01:51.949 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T08:01:51.949 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T08:01:51.950 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T08:01:51.950 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T08:01:51.950 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-16T08:01:51.950 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T08:01:51.950 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T08:01:51.950 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-16T08:01:51.951 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T08:01:51.951 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T08:01:51.951 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-16T08:01:51.951 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-16T08:01:51.951 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T08:01:51.952 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T08:01:51.952 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T08:01:51.952 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T08:01:51.952 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-16T08:01:51.952 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T08:01:51.952 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T08:01:51.953 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T08:01:51.953 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-16T08:01:51.953 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T08:01:51.953 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T08:01:51.953 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T08:01:51.954 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-16T08:01:51.954 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T08:01:51.954 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T08:01:51.954 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T08:01:51.954 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-16T08:01:51.954 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T08:01:51.955 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-16T08:01:51.955 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T08:01:51.955 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-16T08:01:51.955 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T08:01:51.955 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T08:01:51.956 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T08:01:51.956 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T08:01:51.956 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-16T08:01:51.956 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T08:01:51.956 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-16T08:01:51.956 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T08:01:51.957 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T08:01:51.957 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-16T08:01:51.957 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-16T08:01:51.957 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-16T08:01:51.957 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-16T08:01:51.958 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-16T08:01:51.958 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-16T08:01:51.958 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T08:01:51.958 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T08:01:51.958 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-16T08:01:51.959 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T08:01:51.959 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-16T08:01:51.959 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T08:01:51.959 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T08:01:51.959 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-16T08:01:51.959 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T08:01:51.960 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-16T08:01:51.960 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T08:01:51.960 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T08:01:51.960 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T08:01:51.960 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T08:01:51.961 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T08:01:51.961 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T08:01:51.961 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T08:01:51.961 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T08:01:51.961 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T08:01:51.961 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T08:01:51.962 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-16T08:01:51.962 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T08:01:51.962 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T08:01:51.962 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T08:01:51.962 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T08:01:51.963 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-16T08:01:51.963 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-16T08:01:51.963 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T08:01:51.963 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-16T08:01:51.963 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-16T08:01:51.963 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T08:01:51.964 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-16T08:01:51.964 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T08:01:51.964 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T08:01:51.964 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T08:01:51.964 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T08:01:51.965 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T08:01:51.965 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T08:01:51.965 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-16T08:01:51.965 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T08:01:51.965 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-16T08:01:51.965 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-16T08:01:51.966 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-16T08:01:51.966 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T08:01:51.966 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T08:01:51.966 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T08:01:51.966 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T08:01:51.967 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T08:01:51.967 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T08:01:51.967 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T08:01:51.967 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T08:01:51.967 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T08:01:51.967 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T08:01:51.968 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T08:01:51.968 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T08:01:51.968 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-16T08:01:51.968 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T08:01:51.968 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T08:01:51.969 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T08:01:51.969 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-16T08:01:51.969 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-16T08:01:51.969 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T08:01:51.969 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-16T08:01:51.969 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-16T08:01:51.970 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T08:01:51.970 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T08:01:51.970 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T08:01:51.970 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-16T08:01:51.971 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-16T08:01:51.971 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-16T08:01:51.971 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T08:01:51.971 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-16T08:01:51.971 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-16T08:01:51.971 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T08:01:51.972 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-16T08:01:51.972 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-16T08:01:51.972 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T08:01:51.972 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-16T08:01:51.972 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-16T08:01:51.973 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-16T08:01:51.973 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-16T08:01:51.973 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T08:01:51.973 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-16T08:01:51.973 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T08:01:51.974 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-16T08:01:51.974 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-16T08:01:51.974 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T08:01:51.974 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T08:01:51.974 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T08:01:51.974 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-16T08:01:51.975 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-16T08:01:51.975 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-16T08:01:51.975 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T08:01:51.975 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-16T08:01:51.975 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T08:01:51.976 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-16T08:01:51.976 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-16T08:01:51.976 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-16T08:01:51.976 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-16T08:01:51.976 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-16T08:01:51.977 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-16T08:01:51.977 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-16T08:01:51.977 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-16T08:01:51.977 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-16T08:01:51.977 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T08:01:51.977 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T08:01:51.978 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T08:01:51.978 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T08:01:51.978 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-16T08:01:51.978 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-16T08:01:51.978 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-16T08:01:51.979 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-16T08:01:51.979 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-16T08:01:51.979 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-16T08:01:51.979 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-16T08:01:51.979 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-16T08:01:51.980 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-16T08:01:51.980 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-16T08:01:51.980 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-16T08:01:51.980 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-16T08:01:51.980 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-16T08:01:51.980 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-16T08:01:51.981 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-16T08:01:51.981 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-16T08:01:51.981 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-16T08:01:51.981 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-16T08:01:51.981 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-16T08:01:51.981 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-16T08:01:51.982 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-16T08:01:51.982 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-16T08:01:51.982 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-16T08:01:51.982 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-16T08:01:51.982 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-16T08:01:51.983 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-16T08:01:51.983 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-16T08:01:51.983 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-16T08:01:51.983 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-16T08:01:51.983 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-16T08:01:51.983 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-16T08:01:51.984 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-16T08:01:51.984 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-16T08:01:51.984 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-16T08:01:51.984 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-16T08:01:51.984 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-16T08:01:51.985 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-16T08:01:51.985 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-16T08:01:51.985 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-16T08:01:51.985 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-16T08:01:51.985 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-16T08:01:51.985 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-16T08:01:51.986 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-16T08:01:51.986 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-16T08:01:51.986 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-16T08:01:51.986 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-16T08:01:51.986 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-16T08:01:51.987 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-16T08:01:51.987 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-16T08:01:51.987 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-16T08:01:51.987 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-16T08:01:51.987 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-16T08:01:51.987 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-16T08:01:51.988 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-16T08:01:51.988 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-16T08:01:51.988 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-16T08:01:51.988 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-16T08:01:51.988 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-16T08:01:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-16T08:01:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-16T08:01:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-16T08:01:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-16T08:01:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-16T08:01:51.989 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T08:01:51.990 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-16T08:01:51.990 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-16T08:01:51.990 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-16T08:01:51.990 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-16T08:01:51.990 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T08:01:51.991 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T08:01:51.991 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-16T08:01:51.991 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T08:01:51.991 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T08:01:51.991 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-16T08:01:51.992 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-16T08:01:51.992 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-16T08:01:51.992 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T08:01:51.992 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T08:01:51.992 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T08:01:51.992 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T08:01:51.993 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T08:01:51.993 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T08:01:51.993 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T08:01:51.993 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T08:01:51.993 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-16T08:01:51.994 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-16T08:01:51.994 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T08:01:51.994 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T08:01:51.994 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T08:01:51.994 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T08:01:51.995 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T08:01:51.995 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-16T08:01:51.995 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T08:01:51.995 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T08:01:51.995 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-16T08:01:51.995 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-16T08:01:51.996 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-16T08:01:51.996 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-16T08:01:51.996 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-16T08:01:51.996 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T08:01:51.996 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-16T08:01:51.997 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-16T08:01:51.997 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-16T08:01:51.997 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-16T08:01:51.997 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T08:01:51.997 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T08:01:51.997 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T08:01:51.998 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T08:01:51.998 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T08:01:51.998 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T08:01:51.998 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T08:01:51.998 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T08:01:51.999 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T08:01:51.999 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-16T08:01:51.999 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T08:01:51.999 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T08:01:51.999 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T08:01:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T08:01:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T08:01:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T08:01:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T08:01:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T08:01:52.000 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T08:01:52.001 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T08:01:52.001 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T08:01:52.001 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T08:01:52.001 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T08:01:52.001 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-16T08:01:52.002 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T08:01:52.002 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T08:01:52.002 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-16T08:01:52.002 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T08:01:52.002 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T08:01:52.003 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T08:01:52.003 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T08:01:52.003 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T08:01:52.003 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T08:01:52.003 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T08:01:52.003 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T08:01:52.004 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T08:01:52.004 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T08:01:52.004 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T08:01:52.004 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T08:01:52.004 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T08:01:52.005 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T08:01:52.005 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T08:01:52.005 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T08:01:52.005 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T08:01:52.005 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T08:01:52.005 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-16T08:01:52.006 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T08:01:52.006 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T08:01:52.006 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-16T08:01:52.006 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T08:01:52.006 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-16T08:01:52.007 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T08:01:52.007 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T08:01:52.007 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T08:01:52.007 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T08:01:52.007 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T08:01:52.008 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T08:01:52.008 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T08:01:52.008 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T08:01:52.008 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T08:01:52.008 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T08:01:52.008 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T08:01:52.009 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T08:01:52.009 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T08:01:52.009 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T08:01:52.009 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T08:01:52.009 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-16T08:01:52.010 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T08:01:52.010 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T08:01:52.010 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T08:01:52.010 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-16T08:01:52.010 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-16T08:01:52.010 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T08:01:52.011 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T08:01:52.011 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T08:01:52.011 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T08:01:52.011 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-16T08:01:52.011 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T08:01:52.012 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T08:01:52.012 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T08:01:52.012 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T08:01:52.012 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-16T08:01:52.012 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-16T08:01:52.012 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-16T08:01:52.013 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-16T08:01:52.013 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-16T08:01:52.013 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-16T08:01:52.013 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-16T08:01:52.013 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T08:01:52.014 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-16T08:01:52.014 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-16T08:01:52.014 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-16T08:01:52.014 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-16T08:01:52.014 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-16T08:01:52.014 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-16T08:01:52.015 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T08:01:52.015 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-16T08:01:52.015 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-16T08:01:52.015 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-16T08:01:52.015 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-16T08:01:52.016 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-16T08:01:52.016 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-16T08:01:52.016 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T08:01:52.016 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-16T08:01:52.016 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-16T08:01:52.016 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-16T08:01:52.017 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T08:01:52.017 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T08:01:52.017 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-16T08:01:52.017 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-16T08:01:52.017 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-16T08:01:52.017 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T08:01:52.018 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T08:01:52.018 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T08:01:52.018 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T08:01:52.018 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T08:01:52.018 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T08:01:52.019 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T08:01:52.019 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T08:01:52.019 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T08:01:52.019 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T08:01:52.019 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T08:01:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T08:01:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-16T08:01:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-16T08:01:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-16T08:01:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T08:01:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T08:01:52.021 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-16T08:01:52.021 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-16T08:01:52.021 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-16T08:01:52.021 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-16T08:01:52.021 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-16T08:01:52.022 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-16T08:01:52.022 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T08:01:52.022 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T08:01:52.022 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T08:01:52.022 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T08:01:52.023 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T08:01:52.023 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T08:01:52.023 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T08:01:52.023 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T08:01:52.023 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T08:01:52.023 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T08:01:52.024 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-16T08:01:52.024 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T08:01:52.024 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-16T08:01:52.024 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-16T08:01:52.024 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T08:01:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-16T08:01:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-16T08:01:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T08:01:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T08:01:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-16T08:01:52.025 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-16T08:01:52.026 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-16T08:01:52.026 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T08:01:52.026 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-16T08:01:52.026 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T08:01:52.026 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-16T08:01:52.027 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T08:01:52.027 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T08:01:52.027 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-16T08:01:52.027 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T08:01:52.027 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T08:01:52.027 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T08:01:52.028 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T08:01:52.028 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-16T08:01:52.028 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T08:01:52.028 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-16T08:01:52.028 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-16T08:01:52.029 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T08:01:52.029 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-16T08:01:52.029 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T08:01:52.029 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T08:01:52.029 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T08:01:52.029 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-16T08:01:52.030 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T08:01:52.030 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T08:01:52.030 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-16T08:01:52.030 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-16T08:01:52.030 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T08:01:52.031 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-16T08:01:52.031 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-16T08:01:52.031 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T08:01:52.031 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-16T08:01:52.031 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-16T08:01:52.032 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-16T08:01:52.032 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T08:01:52.032 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-16T08:01:52.032 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-16T08:01:52.032 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-16T08:01:52.032 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-16T08:01:52.033 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-16T08:01:52.033 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T08:01:52.033 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-16T08:01:52.033 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-16T08:01:52.033 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T08:01:52.034 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-16T08:01:52.034 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T08:01:52.034 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-16T08:01:52.034 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T08:01:52.034 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-16T08:01:52.035 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T08:01:52.035 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-16T08:01:52.035 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T08:01:52.035 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-16T08:01:52.035 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-16T08:01:52.035 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-16T08:01:52.036 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T08:01:52.036 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-16T08:01:52.036 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T08:01:52.036 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-16T08:01:52.036 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-16T08:01:52.037 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-16T08:01:52.037 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T08:01:52.037 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-16T08:01:52.037 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T08:01:52.037 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-16T08:01:52.037 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-16T08:01:52.038 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-16T08:01:52.038 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T08:01:52.038 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-16T08:01:52.038 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T08:01:52.038 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-16T08:01:52.039 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T08:01:52.039 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-16T08:01:52.039 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T08:01:52.039 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-16T08:01:52.039 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T08:01:52.039 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-16T08:01:52.040 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-16T08:01:52.040 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-16T08:01:52.040 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T08:01:52.040 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-16T08:01:52.040 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T08:01:52.041 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-16T08:01:52.041 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T08:01:52.041 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-16T08:01:52.041 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T08:01:52.041 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-16T08:01:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T08:01:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-16T08:01:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-16T08:01:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T08:01:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-16T08:01:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T08:01:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-16T08:01:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T08:01:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T08:01:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-16T08:01:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T08:01:52.044 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T08:01:52.044 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-16T08:01:52.044 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T08:01:52.044 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T08:01:52.044 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-16T08:01:52.044 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T08:01:52.045 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-16T08:01:52.045 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T08:01:52.045 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-16T08:01:52.045 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T08:01:52.045 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T08:01:52.046 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T08:01:52.046 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T08:01:52.046 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T08:01:52.046 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T08:01:52.046 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T08:01:52.047 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T08:01:52.047 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T08:01:52.047 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T08:01:52.047 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T08:01:52.047 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-16T08:01:52.047 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T08:01:52.048 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T08:01:52.048 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T08:01:52.048 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T08:01:52.048 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T08:01:52.048 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T08:01:52.048 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T08:01:52.049 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T08:01:52.049 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T08:01:52.049 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T08:01:52.049 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T08:01:52.049 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T08:01:52.050 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T08:01:52.050 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T08:01:52.050 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T08:01:52.050 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T08:01:52.050 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T08:01:52.051 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T08:01:52.051 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T08:01:52.051 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T08:01:52.051 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T08:01:52.051 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T08:01:52.052 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T08:01:52.052 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T08:01:52.052 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T08:01:52.052 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T08:01:52.052 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T08:01:52.052 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T08:01:52.053 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T08:01:52.053 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T08:01:52.053 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T08:01:52.053 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T08:01:52.053 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T08:01:52.054 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T08:01:52.054 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T08:01:52.054 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T08:01:52.054 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T08:01:52.054 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T08:01:52.054 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T08:01:52.055 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-16T08:01:52.055 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-16T08:01:52.055 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-16T08:01:52.055 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-16T08:01:52.055 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T08:01:52.056 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-16T08:01:52.056 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T08:01:52.056 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-16T08:01:52.056 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T08:01:52.056 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T08:01:52.056 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T08:01:52.057 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T08:01:52.057 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T08:01:52.057 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-16T08:01:52.057 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-16T08:01:52.057 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-16T08:01:52.058 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T08:01:52.058 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-16T08:01:52.058 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T08:01:52.058 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-16T08:01:52.058 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-16T08:01:52.059 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-16T08:01:52.059 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-16T08:01:52.059 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-16T08:01:52.059 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-16T08:01:52.059 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-16T08:01:52.059 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-16T08:01:52.060 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-16T08:01:52.060 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T08:01:52.060 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-16T08:01:52.060 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T08:01:52.060 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-16T08:01:52.061 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-16T08:01:52.061 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T08:01:52.061 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-16T08:01:52.061 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T08:01:52.061 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T08:01:52.061 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-16T08:01:52.062 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T08:01:52.062 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-16T08:01:52.062 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T08:01:52.062 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-16T08:01:52.062 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T08:01:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-16T08:01:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T08:01:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-16T08:01:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T08:01:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-16T08:01:52.064 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-16T08:01:52.064 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-16T08:01:52.064 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-16T08:01:52.064 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-16T08:01:52.064 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T08:01:52.064 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-16T08:01:52.065 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T08:01:52.065 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-16T08:01:52.065 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T08:01:52.065 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-16T08:01:52.065 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T08:01:52.066 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T08:01:52.066 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-16T08:01:52.066 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T08:01:52.066 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T08:01:52.066 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-16T08:01:52.066 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T08:01:52.067 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T08:01:52.067 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-16T08:01:52.067 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T08:01:52.067 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-16T08:01:52.067 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-16T08:01:52.068 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-16T08:01:52.068 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-16T08:01:52.068 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-16T08:01:52.068 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T08:01:52.068 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-16T08:01:52.069 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-16T08:01:52.069 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T08:01:52.069 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-16T08:01:52.069 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-16T08:01:52.069 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-16T08:01:52.069 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-16T08:01:52.070 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-16T08:01:52.070 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-16T08:01:52.070 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T08:01:52.070 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T08:01:52.070 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T08:01:52.071 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-16T08:01:52.071 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-16T08:01:52.071 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-16T08:01:52.071 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T08:01:52.071 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-16T08:01:52.071 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-16T08:01:52.072 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T08:01:52.072 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T08:01:52.072 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T08:01:52.072 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T08:01:52.072 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T08:01:52.073 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T08:01:52.073 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T08:01:52.073 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T08:01:52.073 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-16T08:01:52.073 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T08:01:52.074 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T08:01:52.074 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T08:01:52.074 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T08:01:52.074 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T08:01:52.074 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-16T08:01:52.074 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T08:01:52.075 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-16T08:01:52.075 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T08:01:52.075 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T08:01:52.075 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T08:01:52.075 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T08:01:52.076 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-16T08:01:52.076 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T08:01:52.076 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-16T08:01:52.076 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T08:01:52.076 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T08:01:52.077 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-16T08:01:52.077 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T08:01:52.077 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T08:01:52.077 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T08:01:52.077 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T08:01:52.077 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T08:01:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T08:01:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T08:01:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T08:01:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T08:01:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T08:01:52.079 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T08:01:52.079 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T08:01:52.079 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T08:01:52.079 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T08:01:52.079 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T08:01:52.080 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T08:01:52.080 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T08:01:52.080 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T08:01:52.080 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T08:01:52.080 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T08:01:52.081 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T08:01:52.081 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T08:01:52.081 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T08:01:52.081 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-16T08:01:52.081 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T08:01:52.082 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-16T08:01:52.082 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T08:01:52.082 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-16T08:01:52.082 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T08:01:52.082 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T08:01:52.082 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T08:01:52.083 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T08:01:52.083 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-16T08:01:52.083 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T08:01:52.083 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T08:01:52.083 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T08:01:52.084 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T08:01:52.084 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T08:01:52.084 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-16T08:01:52.084 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-16T08:01:52.084 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-16T08:01:52.084 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-16T08:01:52.085 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T08:01:52.085 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T08:01:52.085 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T08:01:52.085 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-16T08:01:52.085 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T08:01:52.086 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T08:01:52.086 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T08:01:52.086 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T08:01:52.086 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T08:01:52.086 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T08:01:52.087 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T08:01:52.087 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T08:01:52.087 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T08:01:52.087 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T08:01:52.087 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-16T08:01:52.087 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T08:01:52.088 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T08:01:52.088 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T08:01:52.088 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-16T08:01:52.088 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T08:01:52.088 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-16T08:01:52.089 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T08:01:52.089 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T08:01:52.089 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T08:01:52.089 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-16T08:01:52.089 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T08:01:52.089 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T08:01:52.090 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T08:01:52.090 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T08:01:52.090 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T08:01:52.090 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T08:01:52.090 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T08:01:52.091 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T08:01:52.091 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T08:01:52.091 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-16T08:01:52.091 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T08:01:52.091 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T08:01:52.091 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T08:01:52.092 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T08:01:52.092 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T08:01:52.092 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T08:01:52.092 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T08:01:52.092 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T08:01:52.092 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T08:01:52.093 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T08:01:52.093 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T08:01:52.093 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T08:01:52.093 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T08:01:52.093 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T08:01:52.094 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T08:01:52.094 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T08:01:52.094 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T08:01:52.094 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T08:01:52.094 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T08:01:52.094 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T08:01:52.095 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T08:01:52.095 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T08:01:52.095 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-16T08:01:52.095 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T08:01:52.095 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T08:01:52.096 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T08:01:52.096 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-16T08:01:52.096 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T08:01:52.096 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T08:01:52.096 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T08:01:52.096 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T08:01:52.097 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-16T08:01:52.097 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-16T08:01:52.097 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T08:01:52.097 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-16T08:01:52.097 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T08:01:52.098 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T08:01:52.098 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T08:01:52.098 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T08:01:52.098 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T08:01:52.098 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T08:01:52.099 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T08:01:52.099 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T08:01:52.099 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T08:01:52.099 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T08:01:52.099 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T08:01:52.099 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T08:01:52.100 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T08:01:52.100 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T08:01:52.100 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T08:01:52.100 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T08:01:52.100 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T08:01:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T08:01:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T08:01:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T08:01:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T08:01:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T08:01:52.101 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T08:01:52.102 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T08:01:52.102 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T08:01:52.102 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T08:01:52.102 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-16T08:01:52.102 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T08:01:52.103 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-16T08:01:52.103 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T08:01:52.103 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T08:01:52.103 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T08:01:52.103 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T08:01:52.103 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T08:01:52.104 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T08:01:52.104 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T08:01:52.104 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-16T08:01:52.104 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-16T08:01:52.104 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-16T08:01:52.105 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T08:01:52.105 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-16T08:01:52.105 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-16T08:01:52.105 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T08:01:52.105 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-16T08:01:52.105 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T08:01:52.106 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-16T08:01:52.106 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-16T08:01:52.106 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T08:01:52.106 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T08:01:52.106 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T08:01:52.107 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T08:01:52.107 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T08:01:52.107 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T08:01:52.107 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T08:01:52.107 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T08:01:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T08:01:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T08:01:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T08:01:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T08:01:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T08:01:52.108 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T08:01:52.109 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T08:01:52.109 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T08:01:52.109 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-16T08:01:52.109 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T08:01:52.109 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T08:01:52.110 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T08:01:52.110 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T08:01:52.110 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T08:01:52.110 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T08:01:52.110 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T08:01:52.110 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T08:01:52.111 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T08:01:52.111 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T08:01:52.111 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T08:01:52.111 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T08:01:52.111 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T08:01:52.112 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T08:01:52.112 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T08:01:52.112 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T08:01:52.112 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T08:01:52.112 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T08:01:52.112 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T08:01:52.113 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T08:01:52.113 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T08:01:52.113 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T08:01:52.113 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T08:01:52.113 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T08:01:52.113 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T08:01:52.114 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T08:01:52.114 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T08:01:52.114 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T08:01:52.114 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T08:01:52.114 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T08:01:52.115 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T08:01:52.115 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T08:01:52.115 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T08:01:52.115 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T08:01:52.115 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T08:01:52.115 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T08:01:52.116 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T08:01:52.116 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T08:01:52.116 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T08:01:52.116 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T08:01:52.116 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T08:01:52.116 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T08:01:52.117 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-16T08:01:52.117 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T08:01:52.117 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T08:01:52.117 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T08:01:52.117 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T08:01:52.117 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T08:01:52.118 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T08:01:52.118 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T08:01:52.118 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T08:01:52.118 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T08:01:52.118 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T08:01:52.118 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T08:01:52.119 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T08:01:52.119 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T08:01:52.119 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T08:01:52.119 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T08:01:52.119 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-16T08:01:52.120 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T08:01:52.120 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T08:01:52.120 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T08:01:52.120 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T08:01:52.120 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T08:01:52.121 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-16T08:01:52.121 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-16T08:01:52.121 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-16T08:01:52.121 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T08:01:52.121 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T08:01:52.121 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T08:01:52.122 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T08:01:52.122 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T08:01:52.122 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T08:01:52.122 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T08:01:52.122 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T08:01:52.123 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T08:01:52.123 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T08:01:52.123 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T08:01:52.123 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T08:01:52.123 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T08:01:52.123 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T08:01:52.124 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T08:01:52.124 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T08:01:52.124 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T08:01:52.124 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T08:01:52.124 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T08:01:52.125 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T08:01:52.125 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T08:01:52.125 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T08:01:52.125 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T08:01:52.125 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T08:01:52.126 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T08:01:52.126 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T08:01:52.126 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T08:01:52.126 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T08:01:52.126 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T08:01:52.126 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T08:01:52.127 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T08:01:52.127 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T08:01:52.127 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T08:01:52.127 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T08:01:52.127 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T08:01:52.128 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T08:01:52.128 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T08:01:52.128 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T08:01:52.128 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T08:01:52.128 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T08:01:52.128 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T08:01:52.129 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T08:01:52.129 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T08:01:52.129 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T08:01:52.129 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T08:01:52.129 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T08:01:52.130 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T08:01:52.130 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T08:01:52.130 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T08:01:52.130 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T08:01:52.130 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T08:01:52.130 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T08:01:52.131 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T08:01:52.131 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T08:01:52.131 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T08:01:52.131 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T08:01:52.131 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T08:01:52.132 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T08:01:52.132 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T08:01:52.132 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T08:01:52.132 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T08:01:52.132 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T08:01:52.132 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T08:01:52.133 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T08:01:52.133 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T08:01:52.133 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T08:01:52.133 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T08:01:52.133 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T08:01:52.134 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T08:01:52.134 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-16T08:01:52.134 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T08:01:52.134 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T08:01:52.134 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-16T08:01:52.135 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T08:01:52.135 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-16T08:01:52.135 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T08:01:52.135 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-16T08:01:52.135 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T08:01:52.135 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T08:01:52.136 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T08:01:52.136 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T08:01:52.136 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-16T08:01:52.136 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T08:01:52.136 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-16T08:01:52.137 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-16T08:01:52.137 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-16T08:01:52.137 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-16T08:01:52.137 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T08:01:52.137 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T08:01:52.138 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T08:01:52.138 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-16T08:01:52.138 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-16T08:01:52.138 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-16T08:01:52.138 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T08:01:52.139 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T08:01:52.139 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T08:01:52.139 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T08:01:52.139 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T08:01:52.139 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T08:01:52.139 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T08:01:52.140 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T08:01:52.140 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T08:01:52.140 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T08:01:52.140 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T08:01:52.140 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T08:01:52.141 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-16T08:01:52.141 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T08:01:52.141 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T08:01:52.141 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T08:01:52.141 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T08:01:52.141 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T08:01:52.142 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T08:01:52.142 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T08:01:52.142 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T08:01:52.142 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T08:01:52.142 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T08:01:52.143 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T08:01:52.143 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T08:01:52.143 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T08:01:52.143 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T08:01:52.143 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-16T08:01:52.144 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T08:01:52.144 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T08:01:52.144 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T08:01:52.144 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T08:01:52.144 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T08:01:52.144 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T08:01:52.145 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T08:01:52.145 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T08:01:52.145 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T08:01:52.145 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T08:01:52.145 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T08:01:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T08:01:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T08:01:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T08:01:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T08:01:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T08:01:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T08:01:52.147 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T08:01:52.147 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-16T08:01:52.147 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T08:01:52.147 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T08:01:52.147 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-16T08:01:52.148 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-16T08:01:52.148 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T08:01:52.148 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T08:01:52.148 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T08:01:52.148 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-16T08:01:52.149 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T08:01:52.149 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-16T08:01:52.149 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T08:01:52.149 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T08:01:52.149 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-16T08:01:52.149 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T08:01:52.150 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-16T08:01:52.150 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T08:01:52.150 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T08:01:52.150 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T08:01:52.150 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T08:01:52.151 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T08:01:52.151 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-16T08:01:52.151 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-16T08:01:52.151 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-16T08:01:52.151 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-16T08:01:52.151 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T08:01:52.152 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-16T08:01:52.152 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-16T08:01:52.152 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-16T08:01:52.152 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T08:01:52.152 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T08:01:52.152 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T08:01:52.153 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T08:01:52.153 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-16T08:01:52.153 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T08:01:52.153 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T08:01:52.153 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T08:01:52.154 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T08:01:52.154 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 210 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T08:01:52.154 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T08:01:52.154 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 879 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T08:01:52.154 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T08:01:52.155 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:56 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T08:01:52.155 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/ 2023-10-16T08:01:52.155 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/ 2023-10-16T08:01:52.155 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/ 2023-10-16T08:01:52.155 INFO:teuthology.orchestra.run.smithi186.stdout:drwxr-xr-x root/root 0 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-16T08:01:52.155 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 175 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-16T08:01:52.156 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r-- root/root 691 2023-10-10 19:56 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-16T08:01:52.157 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T08:01:52.157 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-16T08:01:52.157 DEBUG:teuthology.orchestra.run.smithi186:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-16T08:01:52.167 INFO:teuthology.orchestra.run.smithi186.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T08:01:52.167 INFO:teuthology.orchestra.run.smithi186.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T08:01:52.168 INFO:teuthology.orchestra.run.smithi186.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-10-16T08:01:52.168 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-16T08:01:52.169 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-16T08:01:52.169 DEBUG:teuthology.orchestra.run.smithi186:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-16T08:01:52.209 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi186.front.sepia.ceph.com to complete... 2023-10-16T08:01:52.577 INFO:teuthology.orchestra.run.smithi186.stderr:Sourcing file `/etc/default/grub' 2023-10-16T08:01:52.578 INFO:teuthology.orchestra.run.smithi186.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:01:52.607 INFO:teuthology.orchestra.run.smithi186.stderr:Generating grub configuration file ... 2023-10-16T08:01:52.799 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T08:01:52.805 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-16T08:01:53.050 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:01:53.054 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:01:53.094 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:01:53.099 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:01:53.224 INFO:teuthology.orchestra.run.smithi186.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:01:53.225 INFO:teuthology.orchestra.run.smithi186.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:01:53.225 INFO:teuthology.orchestra.run.smithi186.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:01:53.241 INFO:teuthology.orchestra.run.smithi186.stderr:done 2023-10-16T08:02:22.514 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-16T08:02:22.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:02:22.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:02:24.337 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-16T08:02:24.337 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:02:24.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:02:40.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-10-16T08:02:42.682 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-10-16T08:02:49.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:02:49.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:02:51.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:02:51.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:02:54.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-10-16T08:03:06.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:03:06.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:03:08.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-10-16T08:03:20.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:03:20.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:03:23.386 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-10-16T08:03:25.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-10-16T08:03:38.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:03:38.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:03:38.655 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-10-16T08:03:39.391 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:03:39.392 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-rc5-g47672119149b"... 2023-10-16T08:03:39.392 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-10-16T08:03:39.442 INFO:teuthology.orchestra.run.smithi032.stdout:6.6.0-rc5-g47672119149b 2023-10-16T08:03:39.443 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T08:03:39.443 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T08:03:39.443 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-16T08:03:40.180 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:03:40.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:03:40.444 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-16T08:03:40.444 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-16T08:03:40.548 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2023-10-16T08:03:40.563 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-10-16T08:03:40.593 DEBUG:teuthology.parallel:result is None 2023-10-16T08:03:41.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:03:41.744 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-rc5-g47672119149b"... 2023-10-16T08:03:41.745 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2023-10-16T08:03:41.798 INFO:teuthology.orchestra.run.smithi186.stdout:6.6.0-rc5-g47672119149b 2023-10-16T08:03:41.798 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T08:03:41.798 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T08:03:41.799 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-16T08:03:42.800 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-16T08:03:42.801 DEBUG:teuthology.orchestra.run.smithi186:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-16T08:03:43.017 INFO:teuthology.orchestra.run.smithi186.stdout:ttyS1 2023-10-16T08:03:43.057 DEBUG:teuthology.parallel:result is None 2023-10-16T08:03:43.058 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-16T08:03:43.066 INFO:teuthology.task.internal:Creating test directory... 2023-10-16T08:03:43.066 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T08:03:43.070 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T08:03:43.079 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-16T08:03:43.095 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-16T08:03:43.101 INFO:teuthology.task.internal:Creating archive directory... 2023-10-16T08:03:43.101 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T08:03:43.121 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T08:03:43.156 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-16T08:03:43.163 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-16T08:03:43.163 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T08:03:43.197 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T08:03:43.234 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:03:43.248 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:03:43.251 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:03:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:03:43.269 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-16T08:03:43.276 INFO:teuthology.task.internal:Configuring sudo... 2023-10-16T08:03:43.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T08:03:43.281 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T08:03:43.311 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-16T08:03:43.320 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-16T08:03:43.320 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T08:03:43.349 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T08:03:43.362 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T08:03:43.407 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T08:03:43.463 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:03:43.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T08:03:43.538 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T08:03:43.547 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T08:03:43.602 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:03:43.603 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T08:03:43.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-10-16T08:03:43.828 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-10-16T08:03:43.949 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-16T08:03:43.957 INFO:teuthology.task.internal:Starting timer... 2023-10-16T08:03:43.958 INFO:teuthology.run_tasks:Running task pcp... 2023-10-16T08:03:43.970 INFO:teuthology.run_tasks:Running task selinux... 2023-10-16T08:03:43.979 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2023-10-16T08:03:43.980 DEBUG:teuthology.task.selinux:Excluding smithi186: OS 'ubuntu' does not support SELinux 2023-10-16T08:03:43.980 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-16T08:03:43.980 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-16T08:03:43.981 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-16T08:03:43.981 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-16T08:03:43.996 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-16T08:03:43.996 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-16T08:03:44.013 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-16T08:03:44.015 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-16T08:12:24.587 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-10-16T08:12:24.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:12:24.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:12:24.711 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-10-16T08:12:24.818 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-16T08:12:24.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:12:24.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:12:24.908 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-10-16T08:12:25.057 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T08:12:25.057 INFO:teuthology.run_tasks:Running task clock... 2023-10-16T08:12:25.072 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-16T08:12:25.072 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T08:12:25.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:12:25.077 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T08:12:25.077 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:12:25.115 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-16T08:12:25.115 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Command line: ntpd -gq 2023-10-16T08:12:25.115 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: ---------------------------------------------------- 2023-10-16T08:12:25.115 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: ntp-4 is maintained by Network Time Foundation, 2023-10-16T08:12:25.116 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-16T08:12:25.116 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: corporation. Support and training for ntp-4 are 2023-10-16T08:12:25.116 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: available at https://www.nwtime.org/support 2023-10-16T08:12:25.116 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: ---------------------------------------------------- 2023-10-16T08:12:25.117 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: proto: precision = 0.043 usec (-24) 2023-10-16T08:12:25.117 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: basedate set to 2022-02-04 2023-10-16T08:12:25.117 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: gps base set to 2022-02-06 (week 2196) 2023-10-16T08:12:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:12:25.118 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:12:25.119 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:12:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:12:25.120 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listen and drop on 0 v6wildcard [::]:123 2023-10-16T08:12:25.120 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-16T08:12:25.120 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listen normally on 2 lo 127.0.0.1:123 2023-10-16T08:12:25.121 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listen normally on 3 ens1f0 172.21.15.32:123 2023-10-16T08:12:25.121 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listen normally on 4 lo [::1]:123 2023-10-16T08:12:25.121 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%4]:123 2023-10-16T08:12:25.121 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:25 ntpd[27874]: Listening on routing socket on fd #22 for interface updates 2023-10-16T08:12:25.136 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-16T08:12:25.136 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Command line: ntpd -gq 2023-10-16T08:12:25.136 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: ---------------------------------------------------- 2023-10-16T08:12:25.136 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: ntp-4 is maintained by Network Time Foundation, 2023-10-16T08:12:25.137 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-16T08:12:25.137 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: corporation. Support and training for ntp-4 are 2023-10-16T08:12:25.137 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: available at https://www.nwtime.org/support 2023-10-16T08:12:25.137 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: ---------------------------------------------------- 2023-10-16T08:12:25.138 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: proto: precision = 0.042 usec (-24) 2023-10-16T08:12:25.138 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: basedate set to 2022-02-04 2023-10-16T08:12:25.138 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: gps base set to 2022-02-06 (week 2196) 2023-10-16T08:12:25.139 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:12:25.139 INFO:teuthology.orchestra.run.smithi186.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:12:25.139 INFO:teuthology.orchestra.run.smithi186.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:12:25.140 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:12:25.141 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listen and drop on 0 v6wildcard [::]:123 2023-10-16T08:12:25.141 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-16T08:12:25.141 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listen normally on 2 lo 127.0.0.1:123 2023-10-16T08:12:25.141 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listen normally on 3 enp3s0f1 172.21.15.186:123 2023-10-16T08:12:25.141 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listen normally on 4 lo [::1]:123 2023-10-16T08:12:25.142 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6df5%5]:123 2023-10-16T08:12:25.142 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:25 ntpd[27975]: Listening on routing socket on fd #22 for interface updates 2023-10-16T08:12:26.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:26.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:26.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:26.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:26.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:26.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:26.142 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:26.142 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:26 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:28 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:28 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:28 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:28 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:28 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:28 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:28.137 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:28.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:28.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:28.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:28.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:28.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:28.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:28.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:28 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:29 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:29 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:29.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:29 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:29 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:29 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:29 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:30.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:30.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:30.119 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:30 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:30.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:30.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:30.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:30.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:30.141 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:30.142 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:30 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:31.115 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:31 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:31.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:31 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:31.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:31 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:31.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:31 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:31.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:31 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:31.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:31 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:32 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:32.136 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.137 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.137 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.137 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.137 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.137 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:32.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:32.138 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:32.138 INFO:teuthology.orchestra.run.smithi186.stdout:16 Oct 08:12:32 ntpd[27975]: ntpd: time slew -0.001360 s 2023-10-16T08:12:32.139 INFO:teuthology.orchestra.run.smithi186.stdout:ntpd: time slew -0.001360s 2023-10-16T08:12:32.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:32.139 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-16T08:12:32.140 INFO:teuthology.orchestra.run.smithi186.stderr:16 Oct 08:12:32 ntpd[27975]: can't open /var/log/ntpstats/loopstats.20231016: Permission denied 2023-10-16T08:12:32.232 INFO:teuthology.orchestra.run.smithi186.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:12:32.233 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================== 2023-10-16T08:12:32.233 INFO:teuthology.orchestra.run.smithi186.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:32.233 INFO:teuthology.orchestra.run.smithi186.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:32.233 INFO:teuthology.orchestra.run.smithi186.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:32.234 INFO:teuthology.orchestra.run.smithi186.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:33 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:33 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:33 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:33 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:33 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:33 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:12:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:12:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:12:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:12:34.119 INFO:teuthology.orchestra.run.smithi032.stdout:16 Oct 08:12:34 ntpd[27874]: ntpd: time slew -0.002076 s 2023-10-16T08:12:34.120 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew -0.002076s 2023-10-16T08:12:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-16T08:12:34.120 INFO:teuthology.orchestra.run.smithi032.stderr:16 Oct 08:12:34 ntpd[27874]: can't open /var/log/ntpstats/loopstats.20231016: Permission denied 2023-10-16T08:12:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:12:34.218 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2023-10-16T08:12:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:12:34.220 INFO:teuthology.run_tasks:Running task install... 2023-10-16T08:12:34.229 DEBUG:teuthology.task.install:project ceph 2023-10-16T08:12:34.230 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}} 2023-10-16T08:12:34.230 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'} 2023-10-16T08:12:34.231 INFO:teuthology.task.install:Using flavor: default 2023-10-16T08:12:34.237 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-16T08:12:34.237 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-16T08:12:34.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2023-10-16T08:12:34.239 DEBUG:teuthology.orchestra.run.smithi186:> sudo apt-key list | grep Ceph 2023-10-16T08:12:34.347 INFO:teuthology.orchestra.run.smithi186.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-16T08:12:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-16T08:12:34.400 INFO:teuthology.orchestra.run.smithi186.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-16T08:12:34.401 INFO:teuthology.orchestra.run.smithi186.stdout:uid [ unknown] Ceph.com (release key) 2023-10-16T08:12:34.402 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-16T08:12:34.402 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-16T08:12:34.403 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T08:12:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-16T08:12:34.427 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2023-10-16T08:12:34.428 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-16T08:12:34.429 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-16T08:12:34.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T08:12:34.561 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default/ 2023-10-16T08:12:34.562 INFO:teuthology.task.install.deb:Package version is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:12:34.656 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default/ 2023-10-16T08:12:34.656 INFO:teuthology.task.install.deb:Package version is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:12:34.753 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:12:34.754 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-16T08:12:34.776 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:12:34.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-16T08:12:34.777 DEBUG:teuthology.orchestra.run.smithi186:> sudo apt-get update 2023-10-16T08:12:34.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2023-10-16T08:12:34.965 INFO:teuthology.orchestra.run.smithi186.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:12:34.965 INFO:teuthology.orchestra.run.smithi186.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T08:12:34.971 INFO:teuthology.orchestra.run.smithi186.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T08:12:34.988 INFO:teuthology.orchestra.run.smithi186.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T08:12:35.041 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy InRelease 2023-10-16T08:12:35.062 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-16T08:12:35.084 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-16T08:12:35.085 INFO:teuthology.orchestra.run.smithi186.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy InRelease 2023-10-16T08:12:35.105 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-16T08:12:35.120 INFO:teuthology.orchestra.run.smithi186.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-16T08:12:35.128 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:12:35.128 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T08:12:35.149 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-16T08:12:35.158 INFO:teuthology.orchestra.run.smithi186.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-16T08:12:35.190 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T08:12:35.194 INFO:teuthology.orchestra.run.smithi186.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2023-10-16T08:12:35.242 INFO:teuthology.orchestra.run.smithi186.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-16T08:12:35.272 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T08:12:35.570 INFO:teuthology.orchestra.run.smithi186.stdout:Fetched 28.6 kB in 1s (50.3 kB/s) 2023-10-16T08:12:35.755 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 28.6 kB in 1s (39.4 kB/s) 2023-10-16T08:12:36.469 INFO:teuthology.orchestra.run.smithi186.stdout:Reading package lists... 2023-10-16T08:12:36.486 DEBUG:teuthology.orchestra.run.smithi186:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6730-g4ba8a190-1jammy cephadm=18.0.0-6730-g4ba8a190-1jammy ceph-mds=18.0.0-6730-g4ba8a190-1jammy ceph-mgr=18.0.0-6730-g4ba8a190-1jammy ceph-common=18.0.0-6730-g4ba8a190-1jammy ceph-fuse=18.0.0-6730-g4ba8a190-1jammy ceph-test=18.0.0-6730-g4ba8a190-1jammy ceph-volume=18.0.0-6730-g4ba8a190-1jammy radosgw=18.0.0-6730-g4ba8a190-1jammy python3-rados=18.0.0-6730-g4ba8a190-1jammy python3-rgw=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy python3-rbd=18.0.0-6730-g4ba8a190-1jammy libcephfs2=18.0.0-6730-g4ba8a190-1jammy libcephfs-dev=18.0.0-6730-g4ba8a190-1jammy librados2=18.0.0-6730-g4ba8a190-1jammy librbd1=18.0.0-6730-g4ba8a190-1jammy rbd-fuse=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy cephfs-shell=18.0.0-6730-g4ba8a190-1jammy cephfs-top=18.0.0-6730-g4ba8a190-1jammy cephfs-mirror=18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:12:36.561 INFO:teuthology.orchestra.run.smithi186.stdout:Reading package lists... 2023-10-16T08:12:36.686 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-10-16T08:12:36.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6730-g4ba8a190-1jammy cephadm=18.0.0-6730-g4ba8a190-1jammy ceph-mds=18.0.0-6730-g4ba8a190-1jammy ceph-mgr=18.0.0-6730-g4ba8a190-1jammy ceph-common=18.0.0-6730-g4ba8a190-1jammy ceph-fuse=18.0.0-6730-g4ba8a190-1jammy ceph-test=18.0.0-6730-g4ba8a190-1jammy ceph-volume=18.0.0-6730-g4ba8a190-1jammy radosgw=18.0.0-6730-g4ba8a190-1jammy python3-rados=18.0.0-6730-g4ba8a190-1jammy python3-rgw=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy python3-rbd=18.0.0-6730-g4ba8a190-1jammy libcephfs2=18.0.0-6730-g4ba8a190-1jammy libcephfs-dev=18.0.0-6730-g4ba8a190-1jammy librados2=18.0.0-6730-g4ba8a190-1jammy librbd1=18.0.0-6730-g4ba8a190-1jammy rbd-fuse=18.0.0-6730-g4ba8a190-1jammy python3-cephfs=18.0.0-6730-g4ba8a190-1jammy cephfs-shell=18.0.0-6730-g4ba8a190-1jammy cephfs-top=18.0.0-6730-g4ba8a190-1jammy cephfs-mirror=18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:12:36.724 INFO:teuthology.orchestra.run.smithi186.stdout:Building dependency tree... 2023-10-16T08:12:36.725 INFO:teuthology.orchestra.run.smithi186.stdout:Reading state information... 2023-10-16T08:12:36.782 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-10-16T08:12:36.874 INFO:teuthology.orchestra.run.smithi186.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:12:36.874 INFO:teuthology.orchestra.run.smithi186.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:12:36.874 INFO:teuthology.orchestra.run.smithi186.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:12:36.874 INFO:teuthology.orchestra.run.smithi186.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:12:36.875 INFO:teuthology.orchestra.run.smithi186.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:12:36.878 INFO:teuthology.orchestra.run.smithi186.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:12:36.878 INFO:teuthology.orchestra.run.smithi186.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:12:36.878 INFO:teuthology.orchestra.run.smithi186.stdout:The following additional packages will be installed: 2023-10-16T08:12:36.878 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-16T08:12:36.879 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-16T08:12:36.879 INFO:teuthology.orchestra.run.smithi186.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:12:36.879 INFO:teuthology.orchestra.run.smithi186.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-16T08:12:36.879 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-16T08:12:36.880 INFO:teuthology.orchestra.run.smithi186.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-16T08:12:36.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-16T08:12:36.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-16T08:12:36.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-16T08:12:36.880 INFO:teuthology.orchestra.run.smithi186.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-16T08:12:36.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-16T08:12:36.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-16T08:12:36.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-16T08:12:36.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-16T08:12:36.881 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-16T08:12:36.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-16T08:12:36.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-16T08:12:36.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-16T08:12:36.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-16T08:12:36.882 INFO:teuthology.orchestra.run.smithi186.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-16T08:12:36.883 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-16T08:12:36.883 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-16T08:12:36.883 INFO:teuthology.orchestra.run.smithi186.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-16T08:12:36.883 INFO:teuthology.orchestra.run.smithi186.stdout:Suggested packages: 2023-10-16T08:12:36.884 INFO:teuthology.orchestra.run.smithi186.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-16T08:12:36.884 INFO:teuthology.orchestra.run.smithi186.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-16T08:12:36.884 INFO:teuthology.orchestra.run.smithi186.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-16T08:12:36.884 INFO:teuthology.orchestra.run.smithi186.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-16T08:12:36.884 INFO:teuthology.orchestra.run.smithi186.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-16T08:12:36.884 INFO:teuthology.orchestra.run.smithi186.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-16T08:12:36.885 INFO:teuthology.orchestra.run.smithi186.stdout: python3-watchdog 2023-10-16T08:12:36.885 INFO:teuthology.orchestra.run.smithi186.stdout:Recommended packages: 2023-10-16T08:12:36.885 INFO:teuthology.orchestra.run.smithi186.stdout: btrfs-tools 2023-10-16T08:12:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-10-16T08:12:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-10-16T08:12:37.018 INFO:teuthology.orchestra.run.smithi186.stdout:The following NEW packages will be installed: 2023-10-16T08:12:37.019 INFO:teuthology.orchestra.run.smithi186.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-16T08:12:37.019 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-16T08:12:37.019 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-16T08:12:37.020 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-16T08:12:37.020 INFO:teuthology.orchestra.run.smithi186.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:12:37.020 INFO:teuthology.orchestra.run.smithi186.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-16T08:12:37.020 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-16T08:12:37.021 INFO:teuthology.orchestra.run.smithi186.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-16T08:12:37.022 INFO:teuthology.orchestra.run.smithi186.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-16T08:12:37.022 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-16T08:12:37.022 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-16T08:12:37.022 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-16T08:12:37.023 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-16T08:12:37.023 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-16T08:12:37.023 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-16T08:12:37.023 INFO:teuthology.orchestra.run.smithi186.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-16T08:12:37.023 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-16T08:12:37.024 INFO:teuthology.orchestra.run.smithi186.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-16T08:12:37.024 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-16T08:12:37.024 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-16T08:12:37.024 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-16T08:12:37.025 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-16T08:12:37.025 INFO:teuthology.orchestra.run.smithi186.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-16T08:12:37.025 INFO:teuthology.orchestra.run.smithi186.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-16T08:12:37.025 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-16T08:12:37.025 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-16T08:12:37.026 INFO:teuthology.orchestra.run.smithi186.stdout: socat xmlstarlet 2023-10-16T08:12:37.092 INFO:teuthology.orchestra.run.smithi186.stdout:0 upgraded, 115 newly installed, 0 to remove and 259 not upgraded. 2023-10-16T08:12:37.093 INFO:teuthology.orchestra.run.smithi186.stdout:Need to get 184 MB of archives. 2023-10-16T08:12:37.093 INFO:teuthology.orchestra.run.smithi186.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-16T08:12:37.093 INFO:teuthology.orchestra.run.smithi186.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-16T08:12:37.107 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:12:37.107 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:12:37.107 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:12:37.107 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:12:37.108 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:12:37.109 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:12:37.109 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:12:37.110 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-10-16T08:12:37.110 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-16T08:12:37.110 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-16T08:12:37.110 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:12:37.110 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-16T08:12:37.111 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-16T08:12:37.111 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-16T08:12:37.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-16T08:12:37.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-16T08:12:37.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-16T08:12:37.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-16T08:12:37.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-16T08:12:37.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-16T08:12:37.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-16T08:12:37.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-16T08:12:37.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-16T08:12:37.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-16T08:12:37.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-16T08:12:37.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-16T08:12:37.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-16T08:12:37.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-16T08:12:37.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-16T08:12:37.115 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-16T08:12:37.115 INFO:teuthology.orchestra.run.smithi032.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-16T08:12:37.116 INFO:teuthology.orchestra.run.smithi186.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-16T08:12:37.116 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-10-16T08:12:37.116 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-16T08:12:37.116 INFO:teuthology.orchestra.run.smithi032.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-16T08:12:37.116 INFO:teuthology.orchestra.run.smithi032.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-16T08:12:37.117 INFO:teuthology.orchestra.run.smithi032.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-16T08:12:37.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-16T08:12:37.117 INFO:teuthology.orchestra.run.smithi032.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-16T08:12:37.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-watchdog 2023-10-16T08:12:37.117 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2023-10-16T08:12:37.118 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2023-10-16T08:12:37.124 INFO:teuthology.orchestra.run.smithi186.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-16T08:12:37.168 INFO:teuthology.orchestra.run.smithi186.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6730-g4ba8a190-1jammy [3,449 kB] 2023-10-16T08:12:37.174 INFO:teuthology.orchestra.run.smithi186.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-16T08:12:37.184 INFO:teuthology.orchestra.run.smithi186.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-16T08:12:37.203 INFO:teuthology.orchestra.run.smithi186.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-16T08:12:37.205 INFO:teuthology.orchestra.run.smithi186.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-16T08:12:37.217 INFO:teuthology.orchestra.run.smithi186.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-16T08:12:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-10-16T08:12:37.258 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-16T08:12:37.258 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-16T08:12:37.258 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-16T08:12:37.259 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-16T08:12:37.259 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:12:37.259 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-16T08:12:37.259 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-16T08:12:37.261 INFO:teuthology.orchestra.run.smithi032.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-16T08:12:37.261 INFO:teuthology.orchestra.run.smithi032.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-16T08:12:37.261 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-16T08:12:37.261 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-16T08:12:37.261 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-16T08:12:37.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-16T08:12:37.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-16T08:12:37.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-16T08:12:37.262 INFO:teuthology.orchestra.run.smithi032.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-16T08:12:37.263 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-16T08:12:37.263 INFO:teuthology.orchestra.run.smithi032.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-16T08:12:37.263 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-16T08:12:37.263 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-16T08:12:37.263 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-16T08:12:37.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-16T08:12:37.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-16T08:12:37.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-16T08:12:37.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-16T08:12:37.264 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-16T08:12:37.265 INFO:teuthology.orchestra.run.smithi032.stdout: socat xmlstarlet 2023-10-16T08:12:37.265 INFO:teuthology.orchestra.run.smithi186.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-16T08:12:37.265 INFO:teuthology.orchestra.run.smithi186.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-16T08:12:37.274 INFO:teuthology.orchestra.run.smithi186.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-16T08:12:37.277 INFO:teuthology.orchestra.run.smithi186.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-16T08:12:37.278 INFO:teuthology.orchestra.run.smithi186.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-16T08:12:37.285 INFO:teuthology.orchestra.run.smithi186.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-16T08:12:37.286 INFO:teuthology.orchestra.run.smithi186.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-16T08:12:37.287 INFO:teuthology.orchestra.run.smithi186.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-16T08:12:37.288 INFO:teuthology.orchestra.run.smithi186.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-16T08:12:37.288 INFO:teuthology.orchestra.run.smithi186.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-16T08:12:37.307 INFO:teuthology.orchestra.run.smithi186.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-16T08:12:37.308 INFO:teuthology.orchestra.run.smithi186.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-16T08:12:37.308 INFO:teuthology.orchestra.run.smithi186.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-16T08:12:37.309 INFO:teuthology.orchestra.run.smithi186.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-16T08:12:37.309 INFO:teuthology.orchestra.run.smithi186.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-16T08:12:37.327 INFO:teuthology.orchestra.run.smithi186.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-16T08:12:37.328 INFO:teuthology.orchestra.run.smithi186.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-16T08:12:37.328 INFO:teuthology.orchestra.run.smithi186.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-16T08:12:37.329 INFO:teuthology.orchestra.run.smithi186.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-16T08:12:37.329 INFO:teuthology.orchestra.run.smithi186.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-16T08:12:37.346 INFO:teuthology.orchestra.run.smithi186.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-16T08:12:37.347 INFO:teuthology.orchestra.run.smithi186.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-16T08:12:37.348 INFO:teuthology.orchestra.run.smithi186.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-16T08:12:37.349 INFO:teuthology.orchestra.run.smithi186.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-16T08:12:37.349 INFO:teuthology.orchestra.run.smithi186.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-16T08:12:37.366 INFO:teuthology.orchestra.run.smithi186.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-16T08:12:37.368 INFO:teuthology.orchestra.run.smithi186.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-16T08:12:37.368 INFO:teuthology.orchestra.run.smithi186.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-16T08:12:37.369 INFO:teuthology.orchestra.run.smithi186.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-16T08:12:37.369 INFO:teuthology.orchestra.run.smithi186.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-16T08:12:37.386 INFO:teuthology.orchestra.run.smithi186.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-16T08:12:37.386 INFO:teuthology.orchestra.run.smithi186.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-16T08:12:37.388 INFO:teuthology.orchestra.run.smithi186.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-16T08:12:37.389 INFO:teuthology.orchestra.run.smithi186.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-16T08:12:37.391 INFO:teuthology.orchestra.run.smithi186.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-16T08:12:37.403 INFO:teuthology.orchestra.run.smithi186.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6730-g4ba8a190-1jammy [3,130 kB] 2023-10-16T08:12:37.405 INFO:teuthology.orchestra.run.smithi186.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-16T08:12:37.407 INFO:teuthology.orchestra.run.smithi186.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-16T08:12:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 115 newly installed, 0 to remove and 259 not upgraded. 2023-10-16T08:12:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 184 MB of archives. 2023-10-16T08:12:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-16T08:12:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-16T08:12:37.464 INFO:teuthology.orchestra.run.smithi186.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-16T08:12:37.465 INFO:teuthology.orchestra.run.smithi186.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-16T08:12:37.466 INFO:teuthology.orchestra.run.smithi186.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-16T08:12:37.467 INFO:teuthology.orchestra.run.smithi186.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-16T08:12:37.468 INFO:teuthology.orchestra.run.smithi186.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-16T08:12:37.469 INFO:teuthology.orchestra.run.smithi186.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-16T08:12:37.490 INFO:teuthology.orchestra.run.smithi186.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-16T08:12:37.492 INFO:teuthology.orchestra.run.smithi186.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-16T08:12:37.493 INFO:teuthology.orchestra.run.smithi186.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-16T08:12:37.498 INFO:teuthology.orchestra.run.smithi186.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6730-g4ba8a190-1jammy [688 kB] 2023-10-16T08:12:37.511 INFO:teuthology.orchestra.run.smithi186.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-16T08:12:37.511 INFO:teuthology.orchestra.run.smithi186.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-16T08:12:37.512 INFO:teuthology.orchestra.run.smithi186.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-16T08:12:37.512 INFO:teuthology.orchestra.run.smithi186.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-16T08:12:37.513 INFO:teuthology.orchestra.run.smithi186.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-16T08:12:37.514 INFO:teuthology.orchestra.run.smithi186.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-16T08:12:37.515 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6730-g4ba8a190-1jammy [3,449 kB] 2023-10-16T08:12:37.521 INFO:teuthology.orchestra.run.smithi186.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6730-g4ba8a190-1jammy [356 kB] 2023-10-16T08:12:37.525 INFO:teuthology.orchestra.run.smithi186.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-16T08:12:37.526 INFO:teuthology.orchestra.run.smithi186.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-16T08:12:37.530 INFO:teuthology.orchestra.run.smithi186.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-16T08:12:37.533 INFO:teuthology.orchestra.run.smithi186.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6730-g4ba8a190-1jammy [32.8 kB] 2023-10-16T08:12:37.534 INFO:teuthology.orchestra.run.smithi186.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6730-g4ba8a190-1jammy [180 kB] 2023-10-16T08:12:37.538 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-16T08:12:37.540 INFO:teuthology.orchestra.run.smithi186.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6730-g4ba8a190-1jammy [62.4 kB] 2023-10-16T08:12:37.543 INFO:teuthology.orchestra.run.smithi186.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6730-g4ba8a190-1jammy [329 kB] 2023-10-16T08:12:37.545 INFO:teuthology.orchestra.run.smithi186.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-16T08:12:37.547 INFO:teuthology.orchestra.run.smithi186.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-16T08:12:37.548 INFO:teuthology.orchestra.run.smithi186.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-16T08:12:37.552 INFO:teuthology.orchestra.run.smithi186.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-16T08:12:37.554 INFO:teuthology.orchestra.run.smithi186.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6730-g4ba8a190-1jammy [6,222 kB] 2023-10-16T08:12:37.555 INFO:teuthology.orchestra.run.smithi186.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-16T08:12:37.555 INFO:teuthology.orchestra.run.smithi186.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-16T08:12:37.561 INFO:teuthology.orchestra.run.smithi186.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-16T08:12:37.565 INFO:teuthology.orchestra.run.smithi186.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-16T08:12:37.565 INFO:teuthology.orchestra.run.smithi186.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-16T08:12:37.566 INFO:teuthology.orchestra.run.smithi186.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-16T08:12:37.583 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-16T08:12:37.585 INFO:teuthology.orchestra.run.smithi186.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-16T08:12:37.587 INFO:teuthology.orchestra.run.smithi186.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-16T08:12:37.587 INFO:teuthology.orchestra.run.smithi186.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-16T08:12:37.596 INFO:teuthology.orchestra.run.smithi186.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-16T08:12:37.596 INFO:teuthology.orchestra.run.smithi186.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-16T08:12:37.597 INFO:teuthology.orchestra.run.smithi186.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-16T08:12:37.599 INFO:teuthology.orchestra.run.smithi186.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-16T08:12:37.604 INFO:teuthology.orchestra.run.smithi186.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-16T08:12:37.775 INFO:teuthology.orchestra.run.smithi186.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6730-g4ba8a190-1jammy [112 kB] 2023-10-16T08:12:37.776 INFO:teuthology.orchestra.run.smithi186.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6730-g4ba8a190-1jammy [439 kB] 2023-10-16T08:12:37.789 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-16T08:12:37.794 INFO:teuthology.orchestra.run.smithi186.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6730-g4ba8a190-1jammy [22.6 MB] 2023-10-16T08:12:37.833 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-16T08:12:37.909 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-16T08:12:37.920 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-16T08:12:37.966 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-16T08:12:38.132 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-16T08:12:38.141 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-16T08:12:38.167 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-16T08:12:38.340 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-16T08:12:38.533 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-16T08:12:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6730-g4ba8a190-1jammy [6,851 kB] 2023-10-16T08:12:38.848 INFO:teuthology.orchestra.run.smithi186.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6730-g4ba8a190-1jammy [238 kB] 2023-10-16T08:12:38.853 INFO:teuthology.orchestra.run.smithi186.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6730-g4ba8a190-1jammy [128 kB] 2023-10-16T08:12:38.858 INFO:teuthology.orchestra.run.smithi186.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6730-g4ba8a190-1jammy [1,496 kB] 2023-10-16T08:12:38.912 INFO:teuthology.orchestra.run.smithi186.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6730-g4ba8a190-1jammy [9,861 kB] 2023-10-16T08:12:38.950 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-16T08:12:38.958 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-16T08:12:38.960 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-16T08:12:38.971 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-16T08:12:38.971 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-16T08:12:38.973 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-16T08:12:38.975 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-16T08:12:38.976 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-16T08:12:38.978 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-16T08:12:38.999 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6730-g4ba8a190-1jammy [3,130 kB] 2023-10-16T08:12:39.143 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-16T08:12:39.270 INFO:teuthology.orchestra.run.smithi186.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6730-g4ba8a190-1jammy [31.7 MB] 2023-10-16T08:12:39.595 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-16T08:12:39.606 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-16T08:12:39.612 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-16T08:12:39.616 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-16T08:12:39.636 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-16T08:12:39.639 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-16T08:12:39.642 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-16T08:12:39.680 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-16T08:12:39.695 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-16T08:12:39.701 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-16T08:12:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-16T08:12:40.037 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-16T08:12:40.068 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6730-g4ba8a190-1jammy [688 kB] 2023-10-16T08:12:40.127 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-16T08:12:40.205 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-16T08:12:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-16T08:12:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6730-g4ba8a190-1jammy [356 kB] 2023-10-16T08:12:40.395 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-16T08:12:40.414 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-16T08:12:40.422 INFO:teuthology.orchestra.run.smithi186.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6730-g4ba8a190-1jammy [14.0 kB] 2023-10-16T08:12:40.426 INFO:teuthology.orchestra.run.smithi186.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [839 kB] 2023-10-16T08:12:40.438 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6730-g4ba8a190-1jammy [32.8 kB] 2023-10-16T08:12:40.444 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-16T08:12:40.444 INFO:teuthology.orchestra.run.smithi186.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6730-g4ba8a190-1jammy [2,146 kB] 2023-10-16T08:12:40.446 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6730-g4ba8a190-1jammy [180 kB] 2023-10-16T08:12:40.456 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-16T08:12:40.494 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-16T08:12:40.501 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-16T08:12:40.514 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6730-g4ba8a190-1jammy [62.4 kB] 2023-10-16T08:12:40.521 INFO:teuthology.orchestra.run.smithi186.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6730-g4ba8a190-1jammy [283 kB] 2023-10-16T08:12:40.531 INFO:teuthology.orchestra.run.smithi186.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6730-g4ba8a190-1jammy [138 kB] 2023-10-16T08:12:40.532 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6730-g4ba8a190-1jammy [329 kB] 2023-10-16T08:12:40.536 INFO:teuthology.orchestra.run.smithi186.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6730-g4ba8a190-1jammy [1,906 kB] 2023-10-16T08:12:40.607 INFO:teuthology.orchestra.run.smithi186.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6730-g4ba8a190-1jammy [8,625 kB] 2023-10-16T08:12:40.676 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6730-g4ba8a190-1jammy [6,222 kB] 2023-10-16T08:12:40.692 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-16T08:12:40.919 INFO:teuthology.orchestra.run.smithi186.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6730-g4ba8a190-1jammy [14.2 kB] 2023-10-16T08:12:40.920 INFO:teuthology.orchestra.run.smithi186.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6730-g4ba8a190-1jammy [47.8 MB] 2023-10-16T08:12:41.519 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-16T08:12:41.520 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-16T08:12:41.521 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-16T08:12:41.522 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-16T08:12:41.522 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-16T08:12:41.524 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-16T08:12:41.546 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-16T08:12:41.549 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-16T08:12:41.549 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-16T08:12:41.619 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-16T08:12:41.784 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-16T08:12:42.042 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-16T08:12:42.074 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-16T08:12:42.143 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-16T08:12:42.178 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-16T08:12:42.197 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-16T08:12:42.224 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-16T08:12:42.362 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-16T08:12:42.398 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-16T08:12:42.419 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-16T08:12:42.426 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-16T08:12:42.627 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-16T08:12:42.631 INFO:teuthology.orchestra.run.smithi186.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6730-g4ba8a190-1jammy [120 kB] 2023-10-16T08:12:42.632 INFO:teuthology.orchestra.run.smithi186.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6730-g4ba8a190-1jammy [227 kB] 2023-10-16T08:12:42.634 INFO:teuthology.orchestra.run.smithi186.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6730-g4ba8a190-1jammy [33.9 kB] 2023-10-16T08:12:42.635 INFO:teuthology.orchestra.run.smithi186.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6730-g4ba8a190-1jammy [23.4 kB] 2023-10-16T08:12:42.635 INFO:teuthology.orchestra.run.smithi186.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6730-g4ba8a190-1jammy [38.3 kB] 2023-10-16T08:12:42.636 INFO:teuthology.orchestra.run.smithi186.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6730-g4ba8a190-1jammy [12.3 MB] 2023-10-16T08:12:43.069 INFO:teuthology.orchestra.run.smithi186.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [91.6 kB] 2023-10-16T08:12:43.107 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-16T08:12:43.112 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-16T08:12:43.159 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6730-g4ba8a190-1jammy [112 kB] 2023-10-16T08:12:43.185 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6730-g4ba8a190-1jammy [439 kB] 2023-10-16T08:12:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-16T08:12:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6730-g4ba8a190-1jammy [22.6 MB] 2023-10-16T08:12:43.288 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-16T08:12:43.288 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-16T08:12:43.293 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-16T08:12:43.297 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-16T08:12:43.311 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-16T08:12:43.318 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-16T08:12:43.378 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-16T08:12:43.544 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-16T08:12:43.647 INFO:teuthology.orchestra.run.smithi186.stdout:Fetched 184 MB in 6s (30.5 MB/s) 2023-10-16T08:12:43.709 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-16T08:12:43.765 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-16T08:12:43.833 INFO:teuthology.orchestra.run.smithi186.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-16T08:12:43.836 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-16T08:12:43.872 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-16T08:12:44.008 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-16T08:12:44.035 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-16T08:12:44.193 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-16T08:12:44.205 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-16T08:12:44.250 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:12:44.603 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-cmd2. 2023-10-16T08:12:44.617 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-16T08:12:44.662 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-16T08:12:44.981 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-16T08:12:44.996 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-16T08:12:45.081 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:12:45.392 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-16T08:12:45.407 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-16T08:12:45.442 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:12:45.762 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-16T08:12:45.777 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-16T08:12:45.820 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:12:46.107 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-16T08:12:46.122 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-16T08:12:46.165 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:12:46.604 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-16T08:12:46.619 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:12:46.671 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:12:47.031 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6730-g4ba8a190-1jammy [6,851 kB] 2023-10-16T08:12:47.065 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-16T08:12:47.080 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:12:47.115 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:12:47.451 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-16T08:12:47.466 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:12:47.501 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:12:47.838 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-16T08:12:47.853 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-16T08:12:47.896 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:12:47.968 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6730-g4ba8a190-1jammy [238 kB] 2023-10-16T08:12:47.994 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6730-g4ba8a190-1jammy [128 kB] 2023-10-16T08:12:48.020 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6730-g4ba8a190-1jammy [1,496 kB] 2023-10-16T08:12:48.201 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6730-g4ba8a190-1jammy [9,861 kB] 2023-10-16T08:12:48.224 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package librados2. 2023-10-16T08:12:48.239 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../011-librados2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:48.274 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:48.669 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package librbd1. 2023-10-16T08:12:48.683 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../012-librbd1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:48.727 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:49.130 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libcephfs2. 2023-10-16T08:12:49.144 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:49.188 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:49.433 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6730-g4ba8a190-1jammy [31.7 MB] 2023-10-16T08:12:49.457 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-rados. 2023-10-16T08:12:49.471 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:49.515 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:49.860 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-16T08:12:49.874 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:12:49.918 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:50.155 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-cephfs. 2023-10-16T08:12:50.169 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:50.237 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:50.498 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-16T08:12:50.513 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:12:50.556 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:50.893 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-prettytable. 2023-10-16T08:12:50.908 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-16T08:12:50.951 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-16T08:12:51.220 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-rbd. 2023-10-16T08:12:51.235 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:51.279 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:51.640 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-16T08:12:51.655 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-16T08:12:51.690 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:12:51.976 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package librgw2. 2023-10-16T08:12:51.991 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../021-librgw2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:52.026 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:52.478 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-rgw. 2023-10-16T08:12:52.493 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:52.561 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:52.795 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6730-g4ba8a190-1jammy [14.0 kB] 2023-10-16T08:12:52.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [839 kB] 2023-10-16T08:12:52.868 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6730-g4ba8a190-1jammy [2,146 kB] 2023-10-16T08:12:53.069 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6730-g4ba8a190-1jammy [283 kB] 2023-10-16T08:12:53.082 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6730-g4ba8a190-1jammy [138 kB] 2023-10-16T08:12:53.106 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6730-g4ba8a190-1jammy [1,906 kB] 2023-10-16T08:12:53.115 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-16T08:12:53.129 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-16T08:12:53.173 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:12:53.261 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6730-g4ba8a190-1jammy [8,625 kB] 2023-10-16T08:12:53.451 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libradosstriper1. 2023-10-16T08:12:53.465 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:53.500 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:53.965 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6730-g4ba8a190-1jammy [14.2 kB] 2023-10-16T08:12:53.965 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6730-g4ba8a190-1jammy [47.8 MB] 2023-10-16T08:12:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-common. 2023-10-16T08:12:54.236 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:54.279 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:55.376 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-base. 2023-10-16T08:12:55.391 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:12:55.444 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:12:56.089 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-16T08:12:56.103 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-16T08:12:56.147 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:12:56.417 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-cheroot. 2023-10-16T08:12:56.430 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-16T08:12:56.475 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:12:56.578 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6730-g4ba8a190-1jammy [120 kB] 2023-10-16T08:12:56.579 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6730-g4ba8a190-1jammy [227 kB] 2023-10-16T08:12:56.581 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6730-g4ba8a190-1jammy [33.9 kB] 2023-10-16T08:12:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6730-g4ba8a190-1jammy [23.4 kB] 2023-10-16T08:12:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6730-g4ba8a190-1jammy [38.3 kB] 2023-10-16T08:12:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6730-g4ba8a190-1jammy [12.3 MB] 2023-10-16T08:12:56.820 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-16T08:12:56.834 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-16T08:12:56.878 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:12:57.069 INFO:teuthology.orchestra.run.smithi032.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6730-g4ba8a190-1jammy [91.6 kB] 2023-10-16T08:12:57.206 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-16T08:12:57.221 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-16T08:12:57.264 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:12:57.542 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-16T08:12:57.557 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-16T08:12:57.592 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:12:57.680 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 184 MB in 20s (9,294 kB/s) 2023-10-16T08:12:57.832 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-16T08:12:57.853 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-tempora. 2023-10-16T08:12:57.867 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-16T08:12:57.903 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132932 files and directories currently installed.) 2023-10-16T08:12:57.906 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-16T08:12:57.919 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-16T08:12:57.956 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-16T08:12:58.209 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-16T08:12:58.222 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-portend. 2023-10-16T08:12:58.223 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-16T08:12:58.237 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-16T08:12:58.261 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:12:58.280 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-16T08:12:58.570 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cmd2. 2023-10-16T08:12:58.584 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-16T08:12:58.600 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-16T08:12:58.615 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-16T08:12:58.628 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-16T08:12:58.658 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-16T08:12:58.944 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-16T08:12:58.959 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-16T08:12:58.964 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-16T08:12:58.979 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-16T08:12:59.003 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:12:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:12:59.340 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-natsort. 2023-10-16T08:12:59.355 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-16T08:12:59.392 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-16T08:12:59.398 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-16T08:12:59.407 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-16T08:12:59.450 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:12:59.709 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-logutils. 2023-10-16T08:12:59.724 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-16T08:12:59.759 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-16T08:12:59.787 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-16T08:12:59.802 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-16T08:12:59.837 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:13:00.054 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-16T08:13:00.068 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-16T08:13:00.112 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:13:00.265 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-16T08:13:00.280 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-16T08:13:00.323 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:13:00.406 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-mako. 2023-10-16T08:13:00.421 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-16T08:13:00.456 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:13:00.776 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-16T08:13:00.791 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:13:00.801 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-16T08:13:00.817 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-16T08:13:00.843 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:13:00.851 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:13:01.155 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-16T08:13:01.170 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-16T08:13:01.204 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-16T08:13:01.212 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:13:01.219 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:13:01.254 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:13:01.532 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-webob. 2023-10-16T08:13:01.547 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-16T08:13:01.548 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-16T08:13:01.564 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:13:01.590 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:13:01.640 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:13:01.960 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-waitress. 2023-10-16T08:13:01.974 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-16T08:13:01.984 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-16T08:13:02.000 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-16T08:13:02.021 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:13:02.043 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:13:02.354 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-tempita. 2023-10-16T08:13:02.362 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librados2. 2023-10-16T08:13:02.369 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-16T08:13:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-librados2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:02.404 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:13:02.420 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:02.666 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-paste. 2023-10-16T08:13:02.681 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-16T08:13:02.724 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:13:02.840 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librbd1. 2023-10-16T08:13:02.855 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-librbd1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:02.890 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:03.077 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-16T08:13:03.092 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-16T08:13:03.135 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:13:03.275 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2023-10-16T08:13:03.291 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:03.334 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:03.446 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-16T08:13:03.462 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-16T08:13:03.496 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:13:03.620 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2023-10-16T08:13:03.635 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:03.678 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:03.799 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-webtest. 2023-10-16T08:13:03.815 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-16T08:13:03.857 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-16T08:13:03.997 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-16T08:13:04.012 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:04.055 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:04.308 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2023-10-16T08:13:04.323 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:04.369 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pecan. 2023-10-16T08:13:04.384 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-16T08:13:04.401 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:04.427 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:13:04.735 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-16T08:13:04.750 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:04.794 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:04.824 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-certifi. 2023-10-16T08:13:04.839 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-16T08:13:04.882 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-16T08:13:05.130 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2023-10-16T08:13:05.145 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-16T08:13:05.180 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-16T08:13:05.201 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-idna. 2023-10-16T08:13:05.216 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-16T08:13:05.251 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-16T08:13:05.457 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2023-10-16T08:13:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:05.516 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:05.546 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-urllib3. 2023-10-16T08:13:05.561 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-16T08:13:05.604 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:13:05.852 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-16T08:13:05.867 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-16T08:13:05.910 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:13:05.940 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-requests. 2023-10-16T08:13:05.955 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-16T08:13:05.998 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:13:06.254 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2023-10-16T08:13:06.269 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-librgw2_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:06.312 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:06.343 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-16T08:13:06.358 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-16T08:13:06.401 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:13:06.672 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-16T08:13:06.686 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:06.729 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:06.782 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2023-10-16T08:13:06.797 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:06.831 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:07.126 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-16T08:13:07.141 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:07.176 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:07.184 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-16T08:13:07.200 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-16T08:13:07.251 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:13:07.470 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-openssl. 2023-10-16T08:13:07.485 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-16T08:13:07.528 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-16T08:13:07.545 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2023-10-16T08:13:07.560 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:07.603 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:07.865 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mgr. 2023-10-16T08:13:07.880 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:07.923 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2023-10-16T08:13:08.038 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:08.115 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:08.935 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mon. 2023-10-16T08:13:08.950 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:08.993 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:09.204 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2023-10-16T08:13:09.219 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:09.271 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:09.764 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-osd. 2023-10-16T08:13:09.778 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:09.846 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:09.891 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-16T08:13:09.905 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-16T08:13:09.966 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:13:10.902 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cheroot. 2023-10-16T08:13:10.917 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-16T08:13:10.960 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:13:11.041 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph. 2023-10-16T08:13:11.056 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../061-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:11.100 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:11.271 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-16T08:13:11.287 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-16T08:13:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:13:11.452 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-fuse. 2023-10-16T08:13:11.467 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:11.511 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:11.616 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-16T08:13:11.631 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-16T08:13:11.674 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:13:11.847 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mds. 2023-10-16T08:13:11.862 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:11.906 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:12.235 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-16T08:13:12.250 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-16T08:13:12.275 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package cephadm. 2023-10-16T08:13:12.285 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:13:12.289 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../064-cephadm_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:12.358 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:12.571 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempora. 2023-10-16T08:13:12.586 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-16T08:13:12.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-16T08:13:12.670 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-16T08:13:12.685 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-16T08:13:12.728 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-16T08:13:12.934 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-portend. 2023-10-16T08:13:12.949 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-16T08:13:12.992 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-16T08:13:13.057 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-16T08:13:13.071 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:13:13.114 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:13.270 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-16T08:13:13.285 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-16T08:13:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-16T08:13:13.614 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-16T08:13:13.629 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-16T08:13:13.672 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:13:14.061 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2023-10-16T08:13:14.069 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-babel. 2023-10-16T08:13:14.074 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-16T08:13:14.083 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:13:14.117 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-16T08:13:14.119 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:14.412 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2023-10-16T08:13:14.429 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-16T08:13:14.462 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-16T08:13:14.614 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-jinja2. 2023-10-16T08:13:14.628 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-16T08:13:14.672 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-16T08:13:14.756 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-16T08:13:14.771 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-16T08:13:14.814 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:13:14.984 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-16T08:13:14.998 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:15.042 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:15.117 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2023-10-16T08:13:15.132 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-16T08:13:15.175 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:13:15.387 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-16T08:13:15.402 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-16T08:13:15.454 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:13:15.487 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-16T08:13:15.502 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-16T08:13:15.545 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:13:15.765 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-16T08:13:15.780 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-16T08:13:15.823 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-16T08:13:15.872 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-16T08:13:15.888 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-16T08:13:15.922 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:13:16.126 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-routes. 2023-10-16T08:13:16.141 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-16T08:13:16.184 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:13:16.200 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2023-10-16T08:13:16.216 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-16T08:13:16.258 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:13:16.555 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-16T08:13:16.569 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:16.611 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2023-10-16T08:13:16.612 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:16.626 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-16T08:13:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:13:16.980 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2023-10-16T08:13:16.996 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-16T08:13:17.032 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-16T08:13:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:13:17.047 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-16T08:13:17.091 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:13:17.342 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2023-10-16T08:13:17.357 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-16T08:13:17.400 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:13:17.594 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-joblib. 2023-10-16T08:13:17.609 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-16T08:13:17.653 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:13:17.736 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-16T08:13:17.752 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-16T08:13:17.794 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:13:18.015 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-16T08:13:18.029 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-16T08:13:18.065 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:13:18.097 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-16T08:13:18.113 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-16T08:13:18.155 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:13:18.334 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-sklearn. 2023-10-16T08:13:18.349 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-16T08:13:18.392 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:13:18.475 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2023-10-16T08:13:18.490 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-16T08:13:18.533 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-16T08:13:18.844 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2023-10-16T08:13:18.859 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-16T08:13:18.902 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:13:19.101 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-16T08:13:19.115 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:19.185 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:19.265 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-certifi. 2023-10-16T08:13:19.281 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-16T08:13:19.357 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-16T08:13:19.660 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-idna. 2023-10-16T08:13:19.675 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-16T08:13:19.709 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-16T08:13:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-urllib3. 2023-10-16T08:13:19.969 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-16T08:13:20.004 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:13:20.146 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-cachetools. 2023-10-16T08:13:20.161 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-16T08:13:20.204 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-16T08:13:20.307 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests. 2023-10-16T08:13:20.322 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-16T08:13:20.365 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:13:20.482 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-16T08:13:20.497 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-16T08:13:20.532 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:13:20.659 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-16T08:13:20.674 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-16T08:13:20.726 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:13:20.876 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-16T08:13:20.892 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-16T08:13:20.934 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:13:20.996 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-16T08:13:21.011 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:21.237 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-rsa. 2023-10-16T08:13:21.253 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-16T08:13:21.295 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-16T08:13:21.458 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-16T08:13:21.474 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:21.517 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:21.726 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-google-auth. 2023-10-16T08:13:21.741 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-16T08:13:21.784 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-16T08:13:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2023-10-16T08:13:21.835 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-16T08:13:21.877 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-16T08:13:22.129 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-16T08:13:22.144 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-16T08:13:22.187 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:13:22.197 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2023-10-16T08:13:22.212 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:22.247 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:22.499 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-websocket. 2023-10-16T08:13:22.514 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-16T08:13:22.573 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-16T08:13:22.600 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2023-10-16T08:13:22.615 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:22.658 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:22.868 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-16T08:13:22.884 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-16T08:13:22.918 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:13:23.486 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2023-10-16T08:13:23.502 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:23.545 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:23.576 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-16T08:13:23.590 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:23.625 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:23.920 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-16T08:13:23.934 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-16T08:13:23.978 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:13:24.323 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-16T08:13:24.337 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:13:24.381 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:13:24.740 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2023-10-16T08:13:24.755 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-ceph_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:24.790 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:24.842 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package jq. 2023-10-16T08:13:24.857 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:13:24.900 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-16T08:13:25.084 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2023-10-16T08:13:25.099 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:25.142 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:25.170 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package socat. 2023-10-16T08:13:25.185 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-16T08:13:25.220 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:13:25.504 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2023-10-16T08:13:25.519 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:25.562 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:25.573 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package xmlstarlet. 2023-10-16T08:13:25.589 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-16T08:13:25.623 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:13:25.915 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2023-10-16T08:13:25.930 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-cephadm_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:25.973 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:25.977 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-test. 2023-10-16T08:13:25.992 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:26.027 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:26.259 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-16T08:13:26.275 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-16T08:13:26.309 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-16T08:13:26.595 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-16T08:13:26.611 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:13:26.653 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:27.831 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ceph-volume. 2023-10-16T08:13:27.833 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-babel. 2023-10-16T08:13:27.846 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:27.847 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:13:27.883 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:27.889 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:28.220 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2023-10-16T08:13:28.234 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-16T08:13:28.260 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-16T08:13:28.275 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:28.278 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-16T08:13:28.309 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:28.606 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-16T08:13:28.621 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:28.621 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-colorama. 2023-10-16T08:13:28.637 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-16T08:13:28.664 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:28.679 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-16T08:13:28.966 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package cephfs-shell. 2023-10-16T08:13:28.981 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:28.984 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-16T08:13:29.000 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-16T08:13:29.024 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:29.042 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:13:29.268 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package cephfs-top. 2023-10-16T08:13:29.284 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:29.318 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:29.353 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-16T08:13:29.369 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-16T08:13:29.411 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-16T08:13:29.688 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-16T08:13:29.703 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:29.714 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2023-10-16T08:13:29.730 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-16T08:13:29.737 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:29.772 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:13:30.133 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package nvme-cli. 2023-10-16T08:13:30.134 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-16T08:13:30.148 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-16T08:13:30.150 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:30.190 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:13:30.192 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:30.562 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-16T08:13:30.563 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-16T08:13:30.577 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-16T08:13:30.578 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-16T08:13:30.620 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:13:30.621 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:13:31.064 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-16T08:13:31.080 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-16T08:13:31.115 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-16T08:13:31.149 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2023-10-16T08:13:31.165 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-16T08:13:31.207 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:13:31.368 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pastescript. 2023-10-16T08:13:31.384 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-16T08:13:31.425 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-16T08:13:31.586 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-16T08:13:31.601 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-16T08:13:31.644 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:13:31.762 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pluggy. 2023-10-16T08:13:31.778 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-16T08:13:31.820 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:13:31.930 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2023-10-16T08:13:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-16T08:13:31.980 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:13:32.132 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-psutil. 2023-10-16T08:13:32.147 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-16T08:13:32.223 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-16T08:13:32.543 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-py. 2023-10-16T08:13:32.559 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-16T08:13:32.601 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-16T08:13:32.613 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-16T08:13:32.627 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:32.671 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:32.963 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pygments. 2023-10-16T08:13:32.979 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-16T08:13:33.021 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:13:33.467 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-16T08:13:33.481 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-16T08:13:33.516 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:13:33.591 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2023-10-16T08:13:33.609 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-16T08:13:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-16T08:13:33.811 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-toml. 2023-10-16T08:13:33.826 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-16T08:13:33.869 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-16T08:13:33.951 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-16T08:13:33.968 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-16T08:13:34.010 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:13:34.180 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-pytest. 2023-10-16T08:13:34.195 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-16T08:13:34.238 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:13:34.346 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-16T08:13:34.363 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-16T08:13:34.404 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:13:34.592 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package python3-simplejson. 2023-10-16T08:13:34.607 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-16T08:13:34.650 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:13:34.740 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2023-10-16T08:13:34.756 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-16T08:13:34.798 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-16T08:13:34.902 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-16T08:13:34.917 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-16T08:13:34.961 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:13:35.179 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2023-10-16T08:13:35.195 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-16T08:13:35.237 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-16T08:13:35.582 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-16T08:13:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-16T08:13:35.631 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:13:35.715 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package radosgw. 2023-10-16T08:13:35.729 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../113-radosgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:35.772 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:35.909 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2023-10-16T08:13:35.924 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-16T08:13:35.967 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-16T08:13:36.254 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-16T08:13:36.269 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-16T08:13:36.303 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:13:36.367 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package rbd-fuse. 2023-10-16T08:13:36.382 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:36.417 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:36.684 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-16T08:13:36.972 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:13:37.026 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-16T08:13:37.040 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:37.073 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:13:37.083 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:37.320 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-16T08:13:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-16T08:13:37.443 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-16T08:13:37.486 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:13:37.726 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-16T08:13:37.823 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-16T08:13:37.838 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:13:37.881 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:13:38.108 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2023-10-16T08:13:38.124 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:13:38.167 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-16T08:13:38.377 INFO:teuthology.orchestra.run.smithi186.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-16T08:13:38.414 INFO:teuthology.orchestra.run.smithi186.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-16T08:13:38.452 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2023-10-16T08:13:38.468 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-16T08:13:38.484 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:38.511 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:13:38.789 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2023-10-16T08:13:38.805 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-16T08:13:38.847 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:13:39.176 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2023-10-16T08:13:39.190 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:39.259 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:39.289 INFO:teuthology.orchestra.run.smithi186.stdout:Adding system user cephadm....done 2023-10-16T08:13:39.449 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:13:39.766 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:13:40.018 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:13:40.169 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:13:40.428 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-16T08:13:40.730 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:13:40.839 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-16T08:13:41.080 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2023-10-16T08:13:41.096 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:41.132 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:13:41.188 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:41.535 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-16T08:13:41.575 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-16T08:13:41.591 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:41.642 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:41.794 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-16T08:13:41.953 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-colorama. 2023-10-16T08:13:41.970 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-16T08:13:42.011 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-16T08:13:42.037 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-16T08:13:42.281 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-shell. 2023-10-16T08:13:42.296 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:13:42.297 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:42.339 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:42.575 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-top. 2023-10-16T08:13:42.589 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:13:42.590 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6730-g4ba8a190-1jammy_all.deb ... 2023-10-16T08:13:42.633 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:42.857 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:13:43.111 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-16T08:13:43.127 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:43.166 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:43.168 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:43.450 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:13:43.539 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2023-10-16T08:13:43.555 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-16T08:13:43.593 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-16T08:13:43.597 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:13:43.844 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:13:43.977 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-16T08:13:43.992 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-16T08:13:44.034 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:13:44.103 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:13:44.229 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-16T08:13:44.488 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-16T08:13:44.503 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-16T08:13:44.546 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-16T08:13:44.621 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:44.748 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-16T08:13:44.884 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2023-10-16T08:13:44.900 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-16T08:13:44.940 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-16T08:13:45.049 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:13:45.235 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2023-10-16T08:13:45.251 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-16T08:13:45.284 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:13:45.334 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:13:45.604 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2023-10-16T08:13:45.620 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-16T08:13:45.643 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:13:45.662 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-16T08:13:45.990 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2023-10-16T08:13:46.007 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-16T08:13:46.048 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-16T08:13:46.337 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:13:46.402 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pygments. 2023-10-16T08:13:46.418 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-16T08:13:46.459 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:13:46.613 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:13:46.739 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:13:46.889 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:13:46.897 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-16T08:13:46.912 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-16T08:13:46.955 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:13:47.208 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-16T08:13:47.258 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-toml. 2023-10-16T08:13:47.274 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-16T08:13:47.316 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-16T08:13:47.609 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-16T08:13:47.618 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2023-10-16T08:13:47.634 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-16T08:13:47.702 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:13:47.869 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:13:48.005 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2023-10-16T08:13:48.021 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-16T08:13:48.063 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:13:48.237 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:13:48.316 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-16T08:13:48.332 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-16T08:13:48.374 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:13:48.547 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-16T08:13:48.872 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:13:49.077 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2023-10-16T08:13:49.092 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../113-radosgw_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:49.136 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:49.224 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:13:49.492 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:13:49.634 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2023-10-16T08:13:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6730-g4ba8a190-1jammy_amd64.deb ... 2023-10-16T08:13:49.889 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:49.944 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-idna (3.3-1) ... 2023-10-16T08:13:50.190 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-16T08:13:50.287 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:13:50.568 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:13:50.694 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:13:50.963 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-16T08:13:51.374 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-16T08:13:51.517 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:13:51.635 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-16T08:13:51.914 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:13:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-16T08:13:52.020 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:13:52.032 INFO:teuthology.orchestra.run.smithi032.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-10-16T08:13:52.081 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:52.146 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:13:52.447 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:13:52.610 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2023-10-16T08:13:52.749 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-16T08:13:52.770 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:13:53.008 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:13:53.055 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:13:53.159 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:13:53.268 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:13:53.314 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:13:53.465 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:13:53.570 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:13:53.725 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-16T08:13:53.862 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-16T08:13:53.984 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:13:54.110 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-16T08:13:54.138 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-16T08:13:54.428 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:13:54.431 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:13:54.716 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-16T08:13:54.806 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-16T08:13:55.025 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:13:55.099 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-16T08:13:55.143 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-16T08:13:55.350 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-16T08:13:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:13:55.626 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:13:55.678 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:13:55.877 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:13:55.996 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:13:56.203 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:13:56.305 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:13:56.487 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:13:56.598 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:56.747 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:13:56.873 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-16T08:13:57.006 INFO:teuthology.orchestra.run.smithi186.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-16T08:13:57.141 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:13:57.259 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:13:57.360 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:13:57.408 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:13:57.509 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-16T08:13:57.745 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:13:57.861 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:13:58.013 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-16T08:13:58.020 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-16T08:13:58.281 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:13:58.396 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:13:58.406 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-16T08:13:58.681 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:13:58.733 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:13:59.007 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:13:59.067 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:13:59.360 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-16T08:13:59.461 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-16T08:13:59.709 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:13:59.729 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-16T08:13:59.985 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:14:00.013 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:14:00.103 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:14:00.204 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:14:00.281 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:14:00.513 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-16T08:14:00.666 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-16T08:14:00.923 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-16T08:14:00.959 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:14:01.199 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:14:01.328 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:14:01.445 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:01.571 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:01.583 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:14:01.816 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:01.893 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-16T08:14:01.942 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:02.051 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:02.152 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-16T08:14:02.219 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:14:02.579 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:02.587 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:14:02.705 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:02.856 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:02.881 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:14:03.006 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:03.067 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:14:03.067 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:14:03.307 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-idna (3.3-1) ... 2023-10-16T08:14:03.642 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:14:03.699 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:03.842 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:03.993 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:04.094 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:14:04.923 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:14:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-16T08:14:05.300 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:14:05.390 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:05.426 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:14:05.516 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:05.535 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:14:05.667 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:05.768 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:05.845 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:14:05.869 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:05.978 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:06.121 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-16T08:14:06.389 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:14:06.515 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:14:06.533 INFO:teuthology.orchestra.run.smithi186.stdout:Adding group ceph....done 2023-10-16T08:14:06.641 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:14:06.925 INFO:teuthology.orchestra.run.smithi186.stdout:Adding system user ceph....done 2023-10-16T08:14:06.959 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:14:07.111 INFO:teuthology.orchestra.run.smithi186.stdout:Setting system user ceph properties....done 2023-10-16T08:14:07.131 INFO:teuthology.orchestra.run.smithi186.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-16T08:14:07.218 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-16T08:14:07.272 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-16T08:14:07.528 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-16T08:14:07.680 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-16T08:14:07.804 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:14:08.072 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-16T08:14:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:14:08.425 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-16T08:14:08.640 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:14:08.640 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:14:08.850 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:14:08.976 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:14:09.272 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:09.285 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:14:09.399 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:09.603 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:14:09.879 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:14:09.930 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:14:09.931 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:14:10.280 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-16T08:14:10.525 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:10.565 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:14:10.691 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:14:10.854 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-16T08:14:11.084 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:14:11.360 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-16T08:14:11.523 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:11.637 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:14:11.738 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:14:11.738 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:14:11.762 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-16T08:14:12.097 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:14:12.392 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:12.415 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:14:12.588 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:14:12.588 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:14:12.699 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-16T08:14:12.841 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-16T08:14:13.118 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-16T08:14:13.160 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:13.402 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:14:13.454 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:14:13.454 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:14:13.695 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:14:14.086 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:14.122 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-16T08:14:14.212 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:14.354 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:14.431 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:14:14.545 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:14:14.546 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:14:14.792 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:14:14.893 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.010 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.173 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.222 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.315 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.356 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.441 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.482 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.576 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.636 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-16T08:14:15.918 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:15.928 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:14:16.044 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:16.170 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:16.400 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:14:16.400 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:14:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:17.087 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:14:17.156 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:17.274 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-top (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:17.391 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:14:17.568 INFO:teuthology.orchestra.run.smithi186.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:14:17.573 DEBUG:teuthology.orchestra.run.smithi186:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-16T08:14:17.648 INFO:teuthology.orchestra.run.smithi186.stdout:Reading package lists... 2023-10-16T08:14:17.812 INFO:teuthology.orchestra.run.smithi186.stdout:Building dependency tree... 2023-10-16T08:14:17.813 INFO:teuthology.orchestra.run.smithi186.stdout:Reading state information... 2023-10-16T08:14:17.960 INFO:teuthology.orchestra.run.smithi186.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-16T08:14:17.960 INFO:teuthology.orchestra.run.smithi186.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-16T08:14:17.960 INFO:teuthology.orchestra.run.smithi186.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:14:17.961 INFO:teuthology.orchestra.run.smithi186.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:14:17.961 INFO:teuthology.orchestra.run.smithi186.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-16T08:14:17.961 INFO:teuthology.orchestra.run.smithi186.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:14:17.961 INFO:teuthology.orchestra.run.smithi186.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:14:17.961 INFO:teuthology.orchestra.run.smithi186.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:14:17.962 INFO:teuthology.orchestra.run.smithi186.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:14:17.962 INFO:teuthology.orchestra.run.smithi186.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:14:17.963 INFO:teuthology.orchestra.run.smithi186.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:14:17.963 INFO:teuthology.orchestra.run.smithi186.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:14:17.964 INFO:teuthology.orchestra.run.smithi186.stdout:The following additional packages will be installed: 2023-10-16T08:14:17.964 INFO:teuthology.orchestra.run.smithi186.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-16T08:14:17.964 INFO:teuthology.orchestra.run.smithi186.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-16T08:14:17.964 INFO:teuthology.orchestra.run.smithi186.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-16T08:14:17.964 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-16T08:14:17.966 INFO:teuthology.orchestra.run.smithi186.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:14:17.966 INFO:teuthology.orchestra.run.smithi186.stdout:Suggested packages: 2023-10-16T08:14:17.967 INFO:teuthology.orchestra.run.smithi186.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-16T08:14:17.967 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-16T08:14:17.967 INFO:teuthology.orchestra.run.smithi186.stdout: libengine-pkcs11-openssl 2023-10-16T08:14:18.011 INFO:teuthology.orchestra.run.smithi186.stdout:The following NEW packages will be installed: 2023-10-16T08:14:18.013 INFO:teuthology.orchestra.run.smithi186.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-16T08:14:18.013 INFO:teuthology.orchestra.run.smithi186.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-16T08:14:18.013 INFO:teuthology.orchestra.run.smithi186.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-16T08:14:18.013 INFO:teuthology.orchestra.run.smithi186.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-16T08:14:18.013 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-16T08:14:18.014 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-16T08:14:18.014 INFO:teuthology.orchestra.run.smithi186.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:14:18.014 INFO:teuthology.orchestra.run.smithi186.stdout:The following packages will be upgraded: 2023-10-16T08:14:18.015 INFO:teuthology.orchestra.run.smithi186.stdout: libssl3 2023-10-16T08:14:18.101 INFO:teuthology.orchestra.run.smithi186.stdout:1 upgraded, 39 newly installed, 0 to remove and 258 not upgraded. 2023-10-16T08:14:18.101 INFO:teuthology.orchestra.run.smithi186.stdout:Need to get 54.6 MB of archives. 2023-10-16T08:14:18.101 INFO:teuthology.orchestra.run.smithi186.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-16T08:14:18.101 INFO:teuthology.orchestra.run.smithi186.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-16T08:14:18.242 INFO:teuthology.orchestra.run.smithi186.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-16T08:14:18.333 INFO:teuthology.orchestra.run.smithi186.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-16T08:14:18.349 INFO:teuthology.orchestra.run.smithi186.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-16T08:14:18.350 INFO:teuthology.orchestra.run.smithi186.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-16T08:14:18.353 INFO:teuthology.orchestra.run.smithi186.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-16T08:14:18.357 INFO:teuthology.orchestra.run.smithi186.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-16T08:14:18.358 INFO:teuthology.orchestra.run.smithi186.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-16T08:14:18.360 INFO:teuthology.orchestra.run.smithi186.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-16T08:14:18.362 INFO:teuthology.orchestra.run.smithi186.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-16T08:14:18.362 INFO:teuthology.orchestra.run.smithi186.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-16T08:14:18.375 INFO:teuthology.orchestra.run.smithi186.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-16T08:14:18.398 INFO:teuthology.orchestra.run.smithi186.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-16T08:14:18.735 INFO:teuthology.orchestra.run.smithi186.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-16T08:14:18.743 INFO:teuthology.orchestra.run.smithi186.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-16T08:14:18.743 INFO:teuthology.orchestra.run.smithi186.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-16T08:14:18.746 INFO:teuthology.orchestra.run.smithi186.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-16T08:14:18.747 INFO:teuthology.orchestra.run.smithi186.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-16T08:14:18.751 INFO:teuthology.orchestra.run.smithi186.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-16T08:14:18.753 INFO:teuthology.orchestra.run.smithi186.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-16T08:14:18.767 INFO:teuthology.orchestra.run.smithi186.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-16T08:14:18.768 INFO:teuthology.orchestra.run.smithi186.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-16T08:14:18.771 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:18.806 INFO:teuthology.orchestra.run.smithi186.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-16T08:14:18.816 INFO:teuthology.orchestra.run.smithi186.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-16T08:14:18.829 INFO:teuthology.orchestra.run.smithi186.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-16T08:14:18.857 INFO:teuthology.orchestra.run.smithi186.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-16T08:14:18.871 INFO:teuthology.orchestra.run.smithi186.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-16T08:14:18.872 INFO:teuthology.orchestra.run.smithi186.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-16T08:14:18.872 INFO:teuthology.orchestra.run.smithi186.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-16T08:14:18.873 INFO:teuthology.orchestra.run.smithi186.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-16T08:14:18.885 INFO:teuthology.orchestra.run.smithi186.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-16T08:14:18.887 INFO:teuthology.orchestra.run.smithi186.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-16T08:14:18.897 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:19.015 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:19.084 INFO:teuthology.orchestra.run.smithi186.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-16T08:14:19.084 INFO:teuthology.orchestra.run.smithi186.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-16T08:14:19.084 INFO:teuthology.orchestra.run.smithi186.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-16T08:14:19.085 INFO:teuthology.orchestra.run.smithi186.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-16T08:14:19.085 INFO:teuthology.orchestra.run.smithi186.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-16T08:14:19.085 INFO:teuthology.orchestra.run.smithi186.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-16T08:14:19.098 INFO:teuthology.orchestra.run.smithi186.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-16T08:14:19.099 INFO:teuthology.orchestra.run.smithi186.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-16T08:14:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:19.250 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-shell (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:19.409 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:19.479 INFO:teuthology.orchestra.run.smithi186.stdout:Preconfiguring packages ... 2023-10-16T08:14:19.646 INFO:teuthology.orchestra.run.smithi186.stdout:Fetched 54.6 MB in 1s (50.6 MB/s) 2023-10-16T08:14:19.788 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package flex. 2023-10-16T08:14:19.861 INFO:teuthology.orchestra.run.smithi186.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139851 files and directories currently installed.) 2023-10-16T08:14:19.865 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-16T08:14:19.903 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-16T08:14:19.930 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2023-10-16T08:14:20.323 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2023-10-16T08:14:20.481 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:14:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2023-10-16T08:14:20.512 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-16T08:14:20.662 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-16T08:14:20.701 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-16T08:14:21.038 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-16T08:14:21.090 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:14:21.651 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package bison. 2023-10-16T08:14:21.724 INFO:teuthology.orchestra.run.smithi186.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139924 files and directories currently installed.) 2023-10-16T08:14:21.727 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-16T08:14:21.795 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:14:21.870 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-mirror (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:22.085 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:14:22.085 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:14:22.212 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-16T08:14:22.226 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-16T08:14:22.270 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:14:22.615 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package dump. 2023-10-16T08:14:22.630 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-16T08:14:22.665 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking dump (0.4b47-3) ... 2023-10-16T08:14:22.721 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:22.847 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:22.977 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package indent. 2023-10-16T08:14:22.992 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:23.035 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:14:23.287 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-16T08:14:23.303 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-16T08:14:23.346 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:14:23.406 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:14:23.407 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:14:23.682 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-16T08:14:23.698 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-16T08:14:23.741 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:14:23.993 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-16T08:14:24.009 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-16T08:14:24.077 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:14:24.396 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-16T08:14:24.403 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:24.412 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-16T08:14:24.454 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:24.707 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-16T08:14:24.722 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-16T08:14:24.757 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:24.777 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-16T08:14:25.001 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libjson-perl. 2023-10-16T08:14:25.017 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-16T08:14:25.059 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-16T08:14:25.379 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-16T08:14:25.394 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-16T08:14:25.437 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:14:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:25.735 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:14:25.735 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:14:26.365 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:26.585 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:14:26.585 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:14:26.790 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-16T08:14:26.805 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:14:26.848 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:27.126 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-16T08:14:27.141 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:14:27.184 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:27.225 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:27.503 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-16T08:14:27.519 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:14:27.564 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:14:27.564 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:14:27.595 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:27.939 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-16T08:14:27.955 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-16T08:14:27.997 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:14:28.202 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:28.325 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-16T08:14:28.327 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:28.341 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:14:28.375 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:14:28.470 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:28.686 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:14:28.687 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:14:28.736 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-16T08:14:28.752 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:28.794 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:29.130 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-16T08:14:29.146 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:29.189 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:14:29.313 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:29.456 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-16T08:14:29.540 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:29.583 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:14:29.590 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:29.707 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (18.0.0-6730-g4ba8a190-1jammy) ... 2023-10-16T08:14:29.802 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-16T08:14:29.818 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:14:29.868 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:14:30.069 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:14:30.372 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-16T08:14:30.387 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-16T08:14:30.447 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:14:30.800 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package modemmanager. 2023-10-16T08:14:30.815 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:14:30.858 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:31.186 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:14:31.464 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package network-manager. 2023-10-16T08:14:31.480 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:14:31.564 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:14:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:14:31.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-16T08:14:31.776 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-10-16T08:14:31.945 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-10-16T08:14:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-10-16T08:14:32.098 INFO:teuthology.orchestra.run.smithi032.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-16T08:14:32.099 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-16T08:14:32.099 INFO:teuthology.orchestra.run.smithi032.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:14:32.099 INFO:teuthology.orchestra.run.smithi032.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:14:32.100 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-16T08:14:32.100 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:14:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:14:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:14:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:14:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:14:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:14:32.101 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:14:32.102 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-10-16T08:14:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-16T08:14:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-16T08:14:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-16T08:14:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-16T08:14:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:14:32.104 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-10-16T08:14:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-16T08:14:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-16T08:14:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: libengine-pkcs11-openssl 2023-10-16T08:14:32.151 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-10-16T08:14:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-16T08:14:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-16T08:14:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-16T08:14:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-16T08:14:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-16T08:14:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-16T08:14:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:14:32.155 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2023-10-16T08:14:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: libssl3 2023-10-16T08:14:32.187 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package ppp. 2023-10-16T08:14:32.203 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-16T08:14:32.210 INFO:teuthology.orchestra.run.smithi032.stdout:1 upgraded, 39 newly installed, 0 to remove and 258 not upgraded. 2023-10-16T08:14:32.210 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 54.6 MB of archives. 2023-10-16T08:14:32.210 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-16T08:14:32.211 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-16T08:14:32.312 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-16T08:14:32.366 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-16T08:14:32.373 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-16T08:14:32.374 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-16T08:14:32.376 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-16T08:14:32.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-16T08:14:32.377 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-16T08:14:32.378 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-16T08:14:32.379 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-16T08:14:32.380 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-16T08:14:32.385 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-16T08:14:32.397 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-16T08:14:32.575 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:14:32.614 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-16T08:14:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-16T08:14:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-16T08:14:32.619 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-16T08:14:32.619 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-16T08:14:32.624 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-16T08:14:32.625 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-16T08:14:32.633 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-16T08:14:32.633 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-16T08:14:32.655 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-16T08:14:32.655 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-16T08:14:32.665 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-16T08:14:32.686 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-16T08:14:32.690 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-16T08:14:32.690 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-16T08:14:32.691 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-16T08:14:32.691 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-16T08:14:32.703 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-16T08:14:32.705 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-16T08:14:32.852 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-16T08:14:32.852 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-16T08:14:32.852 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-16T08:14:32.852 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-16T08:14:32.853 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-16T08:14:32.853 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-16T08:14:32.866 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-16T08:14:32.866 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-16T08:14:32.933 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package pptp-linux. 2023-10-16T08:14:32.948 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-16T08:14:32.992 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-16T08:14:33.266 INFO:teuthology.orchestra.run.smithi032.stdout:Preconfiguring packages ... 2023-10-16T08:14:33.278 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-16T08:14:33.292 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-16T08:14:33.336 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-16T08:14:33.427 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 54.6 MB in 1s (77.4 MB/s) 2023-10-16T08:14:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package flex. 2023-10-16T08:14:33.631 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-16T08:14:33.646 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-16T08:14:33.660 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139851 files and directories currently installed.) 2023-10-16T08:14:33.663 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-16T08:14:33.689 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql-client-common (238) ... 2023-10-16T08:14:33.708 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-16T08:14:34.151 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-16T08:14:34.167 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:34.218 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:34.236 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:14:34.414 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-16T08:14:34.772 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql-common. 2023-10-16T08:14:34.789 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-16T08:14:34.812 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:14:34.849 INFO:teuthology.orchestra.run.smithi186.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-16T08:14:34.880 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql-common (238) ... 2023-10-16T08:14:35.329 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql-14. 2023-10-16T08:14:35.331 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package bison. 2023-10-16T08:14:35.345 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:35.382 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:35.405 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139924 files and directories currently installed.) 2023-10-16T08:14:35.410 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-16T08:14:35.450 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:14:35.851 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-16T08:14:35.865 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-16T08:14:35.901 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:14:36.254 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package dump. 2023-10-16T08:14:36.269 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-16T08:14:36.312 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking dump (0.4b47-3) ... 2023-10-16T08:14:36.490 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql. 2023-10-16T08:14:36.505 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-16T08:14:36.548 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql (14+238) ... 2023-10-16T08:14:36.591 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package indent. 2023-10-16T08:14:36.606 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:36.649 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:14:36.817 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql-client. 2023-10-16T08:14:36.832 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-16T08:14:36.875 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql-client (14+238) ... 2023-10-16T08:14:36.909 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-16T08:14:36.926 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-16T08:14:36.993 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:14:37.102 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-16T08:14:37.117 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-16T08:14:37.161 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-16T08:14:37.288 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-16T08:14:37.304 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-16T08:14:37.346 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:14:37.505 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package tcl8.6. 2023-10-16T08:14:37.520 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-16T08:14:37.563 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:14:37.607 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-16T08:14:37.623 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-16T08:14:37.657 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:14:37.866 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package tcl. 2023-10-16T08:14:37.882 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-16T08:14:37.921 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-16T08:14:38.002 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-16T08:14:38.018 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-16T08:14:38.060 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:38.277 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package wpasupplicant. 2023-10-16T08:14:38.293 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-16T08:14:38.296 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-16T08:14:38.312 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-16T08:14:38.327 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:14:38.354 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:38.615 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjson-perl. 2023-10-16T08:14:38.631 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-16T08:14:38.664 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-16T08:14:38.673 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-16T08:14:38.679 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-16T08:14:38.722 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-16T08:14:39.010 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-16T08:14:39.026 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-16T08:14:39.060 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:14:39.134 INFO:teuthology.orchestra.run.smithi186.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-16T08:14:39.150 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-16T08:14:39.201 INFO:teuthology.orchestra.run.smithi186.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:14:39.484 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql-client-common (238) ... 2023-10-16T08:14:39.698 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-16T08:14:39.824 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:14:39.950 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:40.076 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:14:40.227 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:14:40.353 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:40.454 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-16T08:14:40.470 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:14:40.512 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:40.773 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-16T08:14:40.789 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:14:40.832 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:41.201 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-16T08:14:41.217 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:14:41.259 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:41.358 INFO:teuthology.orchestra.run.smithi186.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-16T08:14:41.449 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:14:41.553 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-16T08:14:41.569 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-16T08:14:41.603 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:14:41.827 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:14:41.827 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:14:41.963 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-16T08:14:41.979 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:14:42.023 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:14:42.350 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-16T08:14:42.366 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:42.409 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:42.526 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:14:42.637 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:14:42.737 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-16T08:14:42.753 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-16T08:14:42.769 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:42.803 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:14:42.863 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:14:43.022 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:43.105 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-16T08:14:43.121 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:43.148 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:14:43.163 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:14:43.274 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:14:43.432 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-16T08:14:43.448 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:14:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:14:43.913 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:14:44.019 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-16T08:14:44.035 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-16T08:14:44.039 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up dump (0.4b47-3) ... 2023-10-16T08:14:44.077 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:14:44.167 INFO:teuthology.orchestra.run.smithi186.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-16T08:14:44.232 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:44.358 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:14:44.455 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package modemmanager. 2023-10-16T08:14:44.471 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:14:44.484 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-16T08:14:44.513 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:44.610 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-16T08:14:44.770 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:14:44.859 INFO:teuthology.orchestra.run.smithi186.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-16T08:14:44.921 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:14:45.030 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:14:45.153 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager. 2023-10-16T08:14:45.169 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:14:45.181 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:14:45.220 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:14:45.341 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:45.467 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-16T08:14:45.635 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql-client (14+238) ... 2023-10-16T08:14:45.769 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:14:45.835 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ppp. 2023-10-16T08:14:45.851 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-16T08:14:46.207 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:14:46.402 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:14:46.403 INFO:teuthology.orchestra.run.smithi186.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-16T08:14:46.403 INFO:teuthology.orchestra.run.smithi186.stdout:which means they are currently configured by ifupdown: 2023-10-16T08:14:46.403 INFO:teuthology.orchestra.run.smithi186.stdout:- enp3s0f1 2023-10-16T08:14:46.403 INFO:teuthology.orchestra.run.smithi186.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-16T08:14:46.404 INFO:teuthology.orchestra.run.smithi186.stdout:remove their configuration from /etc/network/interfaces. 2023-10-16T08:14:46.404 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:14:46.514 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package pptp-linux. 2023-10-16T08:14:46.529 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-16T08:14:46.535 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-16T08:14:46.596 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-16T08:14:46.852 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-16T08:14:46.867 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-16T08:14:46.905 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-16T08:14:46.910 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-16T08:14:47.189 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-16T08:14:47.204 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-16T08:14:47.238 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-common (238) ... 2023-10-16T08:14:47.287 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-16T08:14:47.643 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-16T08:14:47.659 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:47.701 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:47.947 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:48.123 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:48.231 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-common. 2023-10-16T08:14:48.246 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-16T08:14:48.249 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-16T08:14:48.299 INFO:teuthology.orchestra.run.smithi032.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-16T08:14:48.330 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-common (238) ... 2023-10-16T08:14:48.366 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql-common (238) ... 2023-10-16T08:14:48.805 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-14. 2023-10-16T08:14:48.821 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:14:48.867 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:49.326 INFO:teuthology.orchestra.run.smithi186.stdout:Adding user postgres to group ssl-cert 2023-10-16T08:14:49.573 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:14:49.574 INFO:teuthology.orchestra.run.smithi186.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-16T08:14:49.822 INFO:teuthology.orchestra.run.smithi186.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-16T08:14:49.823 INFO:teuthology.orchestra.run.smithi186.stdout:Removing obsolete dictionary files: 2023-10-16T08:14:49.917 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql. 2023-10-16T08:14:49.932 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-16T08:14:49.975 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql (14+238) ... 2023-10-16T08:14:50.219 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client. 2023-10-16T08:14:50.234 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-16T08:14:50.278 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client (14+238) ... 2023-10-16T08:14:50.523 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-16T08:14:50.546 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-16T08:14:50.562 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-16T08:14:50.613 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-16T08:14:50.882 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package tcl8.6. 2023-10-16T08:14:50.898 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-16T08:14:50.941 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:14:51.009 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:51.227 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package tcl. 2023-10-16T08:14:51.243 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-16T08:14:51.291 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-16T08:14:51.411 INFO:teuthology.orchestra.run.smithi186.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-16T08:14:51.431 INFO:teuthology.orchestra.run.smithi186.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-16T08:14:51.491 INFO:teuthology.orchestra.run.smithi186.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-16T08:14:51.491 INFO:teuthology.orchestra.run.smithi186.stdout:This user must also own the server process. 2023-10-16T08:14:51.492 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:14:51.492 INFO:teuthology.orchestra.run.smithi186.stdout:The database cluster will be initialized with locale "en_US". 2023-10-16T08:14:51.492 INFO:teuthology.orchestra.run.smithi186.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-16T08:14:51.492 INFO:teuthology.orchestra.run.smithi186.stdout:The default text search configuration will be set to "english". 2023-10-16T08:14:51.492 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:14:51.493 INFO:teuthology.orchestra.run.smithi186.stdout:Data page checksums are disabled. 2023-10-16T08:14:51.493 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:14:51.493 INFO:teuthology.orchestra.run.smithi186.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-16T08:14:51.494 INFO:teuthology.orchestra.run.smithi186.stdout:creating subdirectories ... ok 2023-10-16T08:14:51.494 INFO:teuthology.orchestra.run.smithi186.stdout:selecting dynamic shared memory implementation ... posix 2023-10-16T08:14:51.517 INFO:teuthology.orchestra.run.smithi186.stdout:selecting default max_connections ... 100 2023-10-16T08:14:51.551 INFO:teuthology.orchestra.run.smithi186.stdout:selecting default shared_buffers ... 128MB 2023-10-16T08:14:51.565 INFO:teuthology.orchestra.run.smithi186.stdout:selecting default time zone ... Etc/UTC 2023-10-16T08:14:51.566 INFO:teuthology.orchestra.run.smithi186.stdout:creating configuration files ... ok 2023-10-16T08:14:51.672 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package wpasupplicant. 2023-10-16T08:14:51.688 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-16T08:14:51.730 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:14:51.734 INFO:teuthology.orchestra.run.smithi186.stdout:running bootstrap script ... ok 2023-10-16T08:14:52.051 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-16T08:14:52.068 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-16T08:14:52.109 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-16T08:14:52.363 INFO:teuthology.orchestra.run.smithi186.stdout:performing post-bootstrap initialization ... ok 2023-10-16T08:14:52.513 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-16T08:14:52.529 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-16T08:14:52.580 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:14:52.865 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-common (238) ... 2023-10-16T08:14:52.923 INFO:teuthology.orchestra.run.smithi186.stdout:syncing data to disk ... ok 2023-10-16T08:14:53.045 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-16T08:14:53.197 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:14:53.321 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:53.449 INFO:teuthology.orchestra.run.smithi186.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-16T08:14:53.455 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:14:53.581 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:14:53.707 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:14:54.732 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-16T08:14:54.830 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:14:55.243 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:14:55.244 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:14:56.146 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:14:56.318 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:14:56.367 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:14:56.444 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-16T08:14:56.476 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-16T08:14:56.570 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:14:56.627 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up postgresql (14+238) ... 2023-10-16T08:14:56.688 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:14:56.753 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:14:56.797 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:14:56.879 INFO:teuthology.orchestra.run.smithi186.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:56.923 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:14:57.141 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:14:57.141 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:14:57.654 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:14:57.763 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up dump (0.4b47-3) ... 2023-10-16T08:14:57.842 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-16T08:14:57.890 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-16T08:14:57.906 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:14:58.049 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:14:58.175 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-16T08:14:58.216 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:14:58.310 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-16T08:14:58.385 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:14:58.436 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:14:58.534 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-16T08:14:58.604 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:14:58.722 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:14:58.881 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:14:59.032 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:14:59.158 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-16T08:14:59.335 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client (14+238) ... 2023-10-16T08:14:59.461 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:14:59.812 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-16T08:14:59.946 INFO:teuthology.orchestra.run.smithi186.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-16T08:15:00.045 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-16T08:15:00.045 INFO:teuthology.orchestra.run.smithi032.stdout:which means they are currently configured by ifupdown: 2023-10-16T08:15:00.046 INFO:teuthology.orchestra.run.smithi032.stdout:- ens1f0 2023-10-16T08:15:00.046 INFO:teuthology.orchestra.run.smithi032.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-16T08:15:00.046 INFO:teuthology.orchestra.run.smithi032.stdout:remove their configuration from /etc/network/interfaces. 2023-10-16T08:15:00.046 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:00.185 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-16T08:15:00.594 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-16T08:15:00.624 INFO:teuthology.orchestra.run.smithi186.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:15:00.629 DEBUG:teuthology.parallel:result is None 2023-10-16T08:15:01.030 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-16T08:15:01.727 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:15:01.882 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:15:02.008 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-16T08:15:02.134 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-common (238) ... 2023-10-16T08:15:03.101 INFO:teuthology.orchestra.run.smithi032.stdout:Adding user postgres to group ssl-cert 2023-10-16T08:15:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:03.361 INFO:teuthology.orchestra.run.smithi032.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-16T08:15:03.632 INFO:teuthology.orchestra.run.smithi032.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-16T08:15:03.633 INFO:teuthology.orchestra.run.smithi032.stdout:Removing obsolete dictionary files: 2023-10-16T08:15:04.414 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-16T08:15:04.886 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:15:05.311 INFO:teuthology.orchestra.run.smithi032.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-16T08:15:05.326 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-16T08:15:05.380 INFO:teuthology.orchestra.run.smithi032.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-16T08:15:05.381 INFO:teuthology.orchestra.run.smithi032.stdout:This user must also own the server process. 2023-10-16T08:15:05.381 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:05.381 INFO:teuthology.orchestra.run.smithi032.stdout:The database cluster will be initialized with locale "en_US". 2023-10-16T08:15:05.381 INFO:teuthology.orchestra.run.smithi032.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-16T08:15:05.382 INFO:teuthology.orchestra.run.smithi032.stdout:The default text search configuration will be set to "english". 2023-10-16T08:15:05.382 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:05.382 INFO:teuthology.orchestra.run.smithi032.stdout:Data page checksums are disabled. 2023-10-16T08:15:05.382 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:05.382 INFO:teuthology.orchestra.run.smithi032.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-16T08:15:05.383 INFO:teuthology.orchestra.run.smithi032.stdout:creating subdirectories ... ok 2023-10-16T08:15:05.383 INFO:teuthology.orchestra.run.smithi032.stdout:selecting dynamic shared memory implementation ... posix 2023-10-16T08:15:05.407 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default max_connections ... 100 2023-10-16T08:15:05.445 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default shared_buffers ... 128MB 2023-10-16T08:15:05.459 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default time zone ... Etc/UTC 2023-10-16T08:15:05.459 INFO:teuthology.orchestra.run.smithi032.stdout:creating configuration files ... ok 2023-10-16T08:15:05.639 INFO:teuthology.orchestra.run.smithi032.stdout:running bootstrap script ... ok 2023-10-16T08:15:06.291 INFO:teuthology.orchestra.run.smithi032.stdout:performing post-bootstrap initialization ... ok 2023-10-16T08:15:06.937 INFO:teuthology.orchestra.run.smithi032.stdout:syncing data to disk ... ok 2023-10-16T08:15:07.471 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-16T08:15:10.811 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:15:10.929 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-16T08:15:11.089 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql (14+238) ... 2023-10-16T08:15:11.222 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:15:11.348 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:15:11.599 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:15:11.600 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:15:12.385 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-16T08:15:12.720 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:15:12.863 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:15:14.308 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-16T08:15:14.409 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-16T08:15:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:15:15.108 DEBUG:teuthology.parallel:result is None 2023-10-16T08:15:15.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T08:15:15.268 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2023-10-16T08:15:15.291 INFO:teuthology.orchestra.run.smithi032.stdout:18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:15:15.292 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:15:15.292 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730-g4ba8a190-1jammy is installed. 2023-10-16T08:15:15.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T08:15:15.440 DEBUG:teuthology.orchestra.run.smithi186:> dpkg-query -W -f '${Version}' ceph 2023-10-16T08:15:15.464 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:15:15.465 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730-g4ba8a190-1jammy 2023-10-16T08:15:15.465 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730-g4ba8a190-1jammy is installed. 2023-10-16T08:15:15.466 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-16T08:15:15.466 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:15.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T08:15:15.487 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:15.488 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T08:15:15.527 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-16T08:15:15.527 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:15.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T08:15:15.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T08:15:15.612 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:15.613 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T08:15:15.634 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T08:15:15.702 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-16T08:15:15.703 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:15.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T08:15:15.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T08:15:15.784 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:15.785 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T08:15:15.806 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T08:15:15.870 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-16T08:15:15.871 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:15.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T08:15:15.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T08:15:15.952 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:15.952 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T08:15:15.974 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T08:15:16.038 INFO:teuthology.run_tasks:Running task ceph... 2023-10-16T08:15:16.106 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-16T08:15:16.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2023-10-16T08:15:16.111 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /var/log/ceph 2023-10-16T08:15:16.133 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-16T08:15:16.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T08:15:16.172 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T08:15:16.202 INFO:tasks.ceph:Creating extra log directories... 2023-10-16T08:15:16.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T08:15:16.232 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T08:15:16.276 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-16T08:15:16.277 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-16T08:15:16.277 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-16T08:15:16.277 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:16.278 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T08:15:16.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T08:15:16.400 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:16.400 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T08:15:16.410 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T08:15:16.509 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-16T08:15:16.509 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-16T08:15:16.510 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915', 'branch': 'rishabh-2023oct13', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': None, 'first_in_suite': False, 'job_id': '7428915', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'rishabh-2023oct13'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-2023oct13-qa1', 'sha1': '75071e9ab3c3bca3308a5f42b10f82d632c58444'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6006, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'sleep_before_teardown': 0, 'subset': '402/512', 'suite': 'fs', 'suite_branch': 'rishabh-2023oct13-qa1', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '75071e9ab3c3bca3308a5f42b10f82d632c58444', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqvVuFrg37eMep8ddVjkwbT5zCcErGvSYFtwEG/mXOCYAfK/iBsoIRy/HyXeG6IN0IuqYZf+79xZY0ZbxuRTuc=', 'smithi186.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8sPORMgXFVHvX5dfFzOhiDr4AyqKoQe4VOSoiyeAbGE2pFEXowpG0s50XRUwhLwjVPkXWoBj9ftaoMz8BBBHY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_06:54:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-16T08:15:16.511 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T08:15:16.515 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T08:15:16.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T08:15:16.568 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T08:15:16.589 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:16.590 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T08:15:16.636 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T08:15:16.637 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-10-16T08:15:16.693 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T08:15:16.693 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:16.694 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-10-16T08:15:16.694 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:16.694 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-10-16 08:08:44.622617393 +0000 2023-10-16T08:15:16.694 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-10-16 08:08:44.618617429 +0000 2023-10-16T08:15:16.694 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-10-16 08:08:44.618617429 +0000 2023-10-16T08:15:16.695 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-10-16 08:08:44.618617429 +0000 2023-10-16T08:15:16.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T08:15:16.758 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-10-16T08:15:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-10-16T08:15:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000235543 s, 2.2 MB/s 2023-10-16T08:15:16.760 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T08:15:16.815 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-10-16T08:15:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T08:15:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:16.866 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-10-16T08:15:16.866 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:16.866 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-10-16 08:08:45.698607782 +0000 2023-10-16T08:15:16.866 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-10-16 08:08:45.294611391 +0000 2023-10-16T08:15:16.867 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-10-16 08:08:45.294611391 +0000 2023-10-16T08:15:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-10-16 08:08:45.294611391 +0000 2023-10-16T08:15:16.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T08:15:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-10-16T08:15:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-10-16T08:15:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000234612 s, 2.2 MB/s 2023-10-16T08:15:16.929 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T08:15:16.987 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-10-16T08:15:17.041 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T08:15:17.042 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 937 Links: 1 2023-10-16T08:15:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-10-16 08:08:45.998605102 +0000 2023-10-16T08:15:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-10-16 08:08:45.994605137 +0000 2023-10-16T08:15:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-10-16 08:08:45.994605137 +0000 2023-10-16T08:15:17.043 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-10-16 08:08:45.994605137 +0000 2023-10-16T08:15:17.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T08:15:17.107 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-10-16T08:15:17.107 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-10-16T08:15:17.108 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000233395 s, 2.2 MB/s 2023-10-16T08:15:17.109 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T08:15:17.163 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-10-16T08:15:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T08:15:17.217 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:17.217 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 954 Links: 1 2023-10-16T08:15:17.217 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:17.218 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-10-16 08:08:47.126595025 +0000 2023-10-16T08:15:17.218 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-10-16 08:08:46.722598635 +0000 2023-10-16T08:15:17.218 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-10-16 08:08:46.722598635 +0000 2023-10-16T08:15:17.218 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-10-16 08:08:46.722598635 +0000 2023-10-16T08:15:17.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T08:15:17.279 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-10-16T08:15:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-10-16T08:15:17.280 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000235229 s, 2.2 MB/s 2023-10-16T08:15:17.281 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T08:15:17.335 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:17.335 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:17.336 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T08:15:17.346 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T08:15:17.346 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-10-16T08:15:17.398 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T08:15:17.398 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:17.398 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 885 Links: 1 2023-10-16T08:15:17.399 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:17.399 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 08:08:42.255324912 +0000 2023-10-16T08:15:17.399 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 08:08:41.875328306 +0000 2023-10-16T08:15:17.399 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 08:08:41.875328306 +0000 2023-10-16T08:15:17.399 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-10-16 08:08:41.875328306 +0000 2023-10-16T08:15:17.400 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T08:15:17.461 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T08:15:17.461 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T08:15:17.461 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000402366 s, 1.3 MB/s 2023-10-16T08:15:17.463 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T08:15:17.520 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-10-16T08:15:17.574 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T08:15:17.575 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:17.575 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 904 Links: 1 2023-10-16T08:15:17.575 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:17.575 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 08:08:42.911319052 +0000 2023-10-16T08:15:17.575 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 08:08:42.515322588 +0000 2023-10-16T08:15:17.576 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 08:08:42.515322588 +0000 2023-10-16T08:15:17.576 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-10-16 08:08:42.515322588 +0000 2023-10-16T08:15:17.576 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T08:15:17.637 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T08:15:17.637 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T08:15:17.638 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000398308 s, 1.3 MB/s 2023-10-16T08:15:17.639 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T08:15:17.692 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-10-16T08:15:17.742 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T08:15:17.742 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:17.742 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 920 Links: 1 2023-10-16T08:15:17.743 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:17.743 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 08:08:43.603312870 +0000 2023-10-16T08:15:17.743 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 08:08:43.183316623 +0000 2023-10-16T08:15:17.743 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 08:08:43.183316623 +0000 2023-10-16T08:15:17.743 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-10-16 08:08:43.183316623 +0000 2023-10-16T08:15:17.744 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T08:15:17.809 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T08:15:17.809 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T08:15:17.809 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00038624 s, 1.3 MB/s 2023-10-16T08:15:17.811 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T08:15:17.864 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-10-16T08:15:17.914 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T08:15:17.914 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:15:17.915 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 936 Links: 1 2023-10-16T08:15:17.915 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:15:17.915 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 08:08:44.243307155 +0000 2023-10-16T08:15:17.915 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 08:08:43.859310585 +0000 2023-10-16T08:15:17.915 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 08:08:43.859310585 +0000 2023-10-16T08:15:17.916 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2023-10-16 08:08:43.859310585 +0000 2023-10-16T08:15:17.916 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T08:15:17.977 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T08:15:17.977 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T08:15:17.977 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000399562 s, 1.3 MB/s 2023-10-16T08:15:17.979 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T08:15:18.036 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:18.036 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T08:15:18.036 INFO:tasks.ceph:Generating config... 2023-10-16T08:15:18.037 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-16T08:15:18.037 INFO:tasks.ceph:[client] debug client = 20 2023-10-16T08:15:18.038 INFO:tasks.ceph:[client] debug ms = 1 2023-10-16T08:15:18.038 INFO:tasks.ceph:[client] pid file = 2023-10-16T08:15:18.038 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-16T08:15:18.039 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-16T08:15:18.039 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-16T08:15:18.039 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-16T08:15:18.039 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-16T08:15:18.040 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-16T08:15:18.040 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-16T08:15:18.040 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-16T08:15:18.040 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-16T08:15:18.041 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-16T08:15:18.041 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-16T08:15:18.041 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-16T08:15:18.041 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-16T08:15:18.042 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-16T08:15:18.042 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-16T08:15:18.042 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-16T08:15:18.042 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-16T08:15:18.043 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-16T08:15:18.043 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-16T08:15:18.043 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-16T08:15:18.044 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-16T08:15:18.044 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-16T08:15:18.044 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-16T08:15:18.044 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-16T08:15:18.045 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-16T08:15:18.045 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-16T08:15:18.045 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-16T08:15:18.045 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-16T08:15:18.046 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-16T08:15:18.046 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-16T08:15:18.046 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-16T08:15:18.047 INFO:tasks.ceph:Setting up mon.a... 2023-10-16T08:15:18.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-16T08:15:18.095 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2023-10-16T08:15:18.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-16T08:15:18.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T08:15:18.221 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.186'), ('mon.c', '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]')] 2023-10-16T08:15:18.222 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-16T08:15:18.223 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:18.223 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T08:15:18.273 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.186 --addv c '[v2:172.21.15.186:3301,v1:172.21.15.186:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:18.369 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:18.370 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid 943c836a-5c4c-4cb7-a190-9643b3476f99 2023-10-16T08:15:18.370 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2023-10-16T08:15:18.370 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2023-10-16T08:15:18.370 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 943c836a-5c4c-4cb7-a190-9643b3476f99 2023-10-16T08:15:18.370 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2023-10-16T08:15:18.369411+0000 2023-10-16T08:15:18.371 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-10-16T08:15:18.369411+0000 2023-10-16T08:15:18.371 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 16 (pacific) 2023-10-16T08:15:18.371 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2023-10-16T08:15:18.371 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2023-10-16T08:15:18.371 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.b 2023-10-16T08:15:18.372 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] mon.c 2023-10-16T08:15:18.372 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-16T08:15:18.373 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T08:15:18.381 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 943c836a-5c4c-4cb7-a190-9643b3476f99... 2023-10-16T08:15:18.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T08:15:18.431 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T08:15:18.474 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-10-16T08:15:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2023-10-16T08:15:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T08:15:18.474 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2023-10-16T08:15:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2023-10-16T08:15:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2023-10-16T08:15:18.475 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2023-10-16T08:15:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2023-10-16T08:15:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2023-10-16T08:15:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2023-10-16T08:15:18.476 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T08:15:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2023-10-16T08:15:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T08:15:18.477 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T08:15:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2023-10-16T08:15:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2023-10-16T08:15:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2023-10-16T08:15:18.478 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2023-10-16T08:15:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2023-10-16T08:15:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2023-10-16T08:15:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2023-10-16T08:15:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2023-10-16T08:15:18.479 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T08:15:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T08:15:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2023-10-16T08:15:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T08:15:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.480 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2023-10-16T08:15:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2023-10-16T08:15:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2023-10-16T08:15:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2023-10-16T08:15:18.481 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2023-10-16T08:15:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2023-10-16T08:15:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 943c836a-5c4c-4cb7-a190-9643b3476f99 2023-10-16T08:15:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.482 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2023-10-16T08:15:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2023-10-16T08:15:18.482 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2023-10-16T08:15:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2023-10-16T08:15:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2023-10-16T08:15:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.483 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2023-10-16T08:15:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2023-10-16T08:15:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2023-10-16T08:15:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2023-10-16T08:15:18.484 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2023-10-16T08:15:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2023-10-16T08:15:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2023-10-16T08:15:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2023-10-16T08:15:18.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2023-10-16T08:15:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2023-10-16T08:15:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2023-10-16T08:15:18.486 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2023-10-16T08:15:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2023-10-16T08:15:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2023-10-16T08:15:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2023-10-16T08:15:18.487 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2023-10-16T08:15:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2023-10-16T08:15:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2023-10-16T08:15:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-10-16T08:15:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2023-10-16T08:15:18.488 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2023-10-16T08:15:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T08:15:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2023-10-16T08:15:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2023-10-16T08:15:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2023-10-16T08:15:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2023-10-16T08:15:18.489 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-10-16T08:15:18.490 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.490 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2023-10-16T08:15:18.490 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-10-16T08:15:18.490 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2023-10-16T08:15:18.490 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-10-16T08:15:18.491 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-10-16T08:15:18.491 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:15:18.491 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:15:18.491 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2023-10-16T08:15:18.491 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.491 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2023-10-16T08:15:18.492 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-10-16T08:15:18.492 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-10-16T08:15:18.492 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2023-10-16T08:15:18.492 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-10-16T08:15:18.492 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2023-10-16T08:15:18.492 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2023-10-16T08:15:18.493 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:15:18.493 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2023-10-16T08:15:18.493 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2023-10-16T08:15:18.493 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:15:18.493 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.493 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T08:15:18.494 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T08:15:18.494 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T08:15:18.494 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.494 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T08:15:18.494 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T08:15:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T08:15:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2023-10-16T08:15:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2023-10-16T08:15:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.495 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2023-10-16T08:15:18.496 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2023-10-16T08:15:18.496 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:18.496 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2023-10-16T08:15:18.496 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2023-10-16T08:15:18.496 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2023-10-16T08:15:18.496 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2023-10-16T08:15:18.497 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T08:15:18.497 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T08:15:18.497 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2023-10-16T08:15:18.497 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-10-16T08:15:18.497 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-10-16T08:15:18.497 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2023-10-16T08:15:18.498 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-10-16T08:15:18.498 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-10-16T08:15:18.498 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2023-10-16T08:15:18.498 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2023-10-16T08:15:18.498 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2023-10-16T08:15:18.499 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-10-16T08:15:18.499 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-10-16T08:15:18.499 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-10-16T08:15:18.499 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-10-16T08:15:18.499 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2023-10-16T08:15:18.499 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2023-10-16T08:15:18.500 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2023-10-16T08:15:18.500 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-10-16T08:15:18.500 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2023-10-16T08:15:18.500 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2023-10-16T08:15:18.500 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2023-10-16T08:15:18.500 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2023-10-16T08:15:18.501 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-10-16T08:15:18.501 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-10-16T08:15:18.501 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-10-16T08:15:18.502 INFO:teuthology.orchestra.run.smithi186.stdout:[global] 2023-10-16T08:15:18.503 INFO:teuthology.orchestra.run.smithi186.stdout: chdir = "" 2023-10-16T08:15:18.503 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T08:15:18.503 INFO:teuthology.orchestra.run.smithi186.stdout: auth supported = cephx 2023-10-16T08:15:18.503 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.503 INFO:teuthology.orchestra.run.smithi186.stdout: filestore xattr use omap = true 2023-10-16T08:15:18.504 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.504 INFO:teuthology.orchestra.run.smithi186.stdout: mon clock drift allowed = 1.000 2023-10-16T08:15:18.504 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.504 INFO:teuthology.orchestra.run.smithi186.stdout: osd crush chooseleaf type = 0 2023-10-16T08:15:18.504 INFO:teuthology.orchestra.run.smithi186.stdout: auth debug = true 2023-10-16T08:15:18.504 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.505 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on old message = true 2023-10-16T08:15:18.505 INFO:teuthology.orchestra.run.smithi186.stdout: ms die on bug = true 2023-10-16T08:15:18.505 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.505 INFO:teuthology.orchestra.run.smithi186.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T08:15:18.505 INFO:teuthology.orchestra.run.smithi186.stdout: mon pg warn max object skew = 0 2023-10-16T08:15:18.506 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.506 INFO:teuthology.orchestra.run.smithi186.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T08:15:18.506 INFO:teuthology.orchestra.run.smithi186.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T08:15:18.506 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.506 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default size = 2 2023-10-16T08:15:18.507 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.507 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow primary affinity = true 2023-10-16T08:15:18.507 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd allow pg remap = true 2023-10-16T08:15:18.507 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on legacy crush tunables = false 2023-10-16T08:15:18.507 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on crush straw calc version zero = false 2023-10-16T08:15:18.507 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on no sortbitwise = false 2023-10-16T08:15:18.508 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on osd down out interval zero = false 2023-10-16T08:15:18.508 INFO:teuthology.orchestra.run.smithi186.stdout: mon warn on too few osds = false 2023-10-16T08:15:18.508 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T08:15:18.508 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T08:15:18.508 INFO:teuthology.orchestra.run.smithi186.stdout: mon_allow_pool_size_one = true 2023-10-16T08:15:18.508 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.509 INFO:teuthology.orchestra.run.smithi186.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T08:15:18.509 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.509 INFO:teuthology.orchestra.run.smithi186.stdout: osd default data pool replay window = 5 2023-10-16T08:15:18.509 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.509 INFO:teuthology.orchestra.run.smithi186.stdout: mon allow pool delete = true 2023-10-16T08:15:18.510 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.510 INFO:teuthology.orchestra.run.smithi186.stdout: mon cluster log file level = debug 2023-10-16T08:15:18.510 INFO:teuthology.orchestra.run.smithi186.stdout: debug asserts on shutdown = true 2023-10-16T08:15:18.510 INFO:teuthology.orchestra.run.smithi186.stdout: mon health detail to clog = false 2023-10-16T08:15:18.510 INFO:teuthology.orchestra.run.smithi186.stdout: mon host = "172.21.15.32,172.21.15.186,[v2:172.21.15.186:3301,v1:172.21.15.186:6790]" 2023-10-16T08:15:18.510 INFO:teuthology.orchestra.run.smithi186.stdout: fsid = 943c836a-5c4c-4cb7-a190-9643b3476f99 2023-10-16T08:15:18.511 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.511 INFO:teuthology.orchestra.run.smithi186.stdout:[osd] 2023-10-16T08:15:18.511 INFO:teuthology.orchestra.run.smithi186.stdout: osd journal size = 100 2023-10-16T08:15:18.511 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.511 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub load threshold = 5.0 2023-10-16T08:15:18.512 INFO:teuthology.orchestra.run.smithi186.stdout: osd scrub max interval = 600 2023-10-16T08:15:18.512 INFO:teuthology.orchestra.run.smithi186.stdout: osd mclock profile = high_recovery_ops 2023-10-16T08:15:18.512 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.512 INFO:teuthology.orchestra.run.smithi186.stdout: osd recover clone overlap = true 2023-10-16T08:15:18.512 INFO:teuthology.orchestra.run.smithi186.stdout: osd recovery max chunk = 1048576 2023-10-16T08:15:18.513 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.513 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug shutdown = true 2023-10-16T08:15:18.513 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug op order = true 2023-10-16T08:15:18.513 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug verify stray on activate = true 2023-10-16T08:15:18.513 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.513 INFO:teuthology.orchestra.run.smithi186.stdout: osd open classes on start = true 2023-10-16T08:15:18.514 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug pg log writeout = true 2023-10-16T08:15:18.514 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.514 INFO:teuthology.orchestra.run.smithi186.stdout: osd deep scrub update digest min age = 30 2023-10-16T08:15:18.514 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.514 INFO:teuthology.orchestra.run.smithi186.stdout: osd map max advance = 10 2023-10-16T08:15:18.515 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.515 INFO:teuthology.orchestra.run.smithi186.stdout: journal zero on create = true 2023-10-16T08:15:18.515 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.515 INFO:teuthology.orchestra.run.smithi186.stdout: filestore ondisk finisher threads = 3 2023-10-16T08:15:18.515 INFO:teuthology.orchestra.run.smithi186.stdout: filestore apply finisher threads = 3 2023-10-16T08:15:18.516 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.516 INFO:teuthology.orchestra.run.smithi186.stdout: bdev debug aio = true 2023-10-16T08:15:18.516 INFO:teuthology.orchestra.run.smithi186.stdout: osd debug misdirected ops = true 2023-10-16T08:15:18.516 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore block size = 96636764160 2023-10-16T08:15:18.516 INFO:teuthology.orchestra.run.smithi186.stdout: bluestore fsck on mount = True 2023-10-16T08:15:18.516 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluefs = 20 2023-10-16T08:15:18.517 INFO:teuthology.orchestra.run.smithi186.stdout: debug bluestore = 20 2023-10-16T08:15:18.517 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-10-16T08:15:18.517 INFO:teuthology.orchestra.run.smithi186.stdout: debug osd = 20 2023-10-16T08:15:18.517 INFO:teuthology.orchestra.run.smithi186.stdout: debug rocksdb = 10 2023-10-16T08:15:18.517 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T08:15:18.518 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd full ratio = 0.9 2023-10-16T08:15:18.518 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd nearfull ratio = 0.8 2023-10-16T08:15:18.518 INFO:teuthology.orchestra.run.smithi186.stdout: osd failsafe full ratio = 0.95 2023-10-16T08:15:18.518 INFO:teuthology.orchestra.run.smithi186.stdout: osd objectstore = bluestore 2023-10-16T08:15:18.518 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2023-10-16T08:15:18.518 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.519 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr] 2023-10-16T08:15:18.519 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-10-16T08:15:18.519 INFO:teuthology.orchestra.run.smithi186.stdout: debug mgr = 20 2023-10-16T08:15:18.519 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-10-16T08:15:18.519 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-10-16T08:15:18.519 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:15:18.520 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:15:18.520 INFO:teuthology.orchestra.run.smithi186.stdout: mgr/telemetry/nag = false 2023-10-16T08:15:18.520 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.520 INFO:teuthology.orchestra.run.smithi186.stdout:[mon] 2023-10-16T08:15:18.520 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-10-16T08:15:18.521 INFO:teuthology.orchestra.run.smithi186.stdout: debug mon = 20 2023-10-16T08:15:18.521 INFO:teuthology.orchestra.run.smithi186.stdout: debug paxos = 20 2023-10-16T08:15:18.521 INFO:teuthology.orchestra.run.smithi186.stdout: debug auth = 20 2023-10-16T08:15:18.521 INFO:teuthology.orchestra.run.smithi186.stdout: mon data avail warn = 5 2023-10-16T08:15:18.521 INFO:teuthology.orchestra.run.smithi186.stdout: mon mgr mkfs grace = 240 2023-10-16T08:15:18.521 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:15:18.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd reporter subtree level = osd 2023-10-16T08:15:18.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon osd prime pg temp = true 2023-10-16T08:15:18.522 INFO:teuthology.orchestra.run.smithi186.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:15:18.522 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.522 INFO:teuthology.orchestra.run.smithi186.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T08:15:18.523 INFO:teuthology.orchestra.run.smithi186.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T08:15:18.523 INFO:teuthology.orchestra.run.smithi186.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T08:15:18.523 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.523 INFO:teuthology.orchestra.run.smithi186.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T08:15:18.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T08:15:18.523 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T08:15:18.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.524 INFO:teuthology.orchestra.run.smithi186.stdout: # 1m isn't quite enough 2023-10-16T08:15:18.524 INFO:teuthology.orchestra.run.smithi186.stdout: mon_down_mkfs_grace = 2m 2023-10-16T08:15:18.524 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.524 INFO:teuthology.orchestra.run.smithi186.stdout: mon_warn_on_filestore_osds = false 2023-10-16T08:15:18.524 INFO:teuthology.orchestra.run.smithi186.stdout: mon op complaint time = 120 2023-10-16T08:15:18.525 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T08:15:18.525 INFO:teuthology.orchestra.run.smithi186.stdout:[client] 2023-10-16T08:15:18.525 INFO:teuthology.orchestra.run.smithi186.stdout: rgw cache enabled = true 2023-10-16T08:15:18.525 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable ops log = true 2023-10-16T08:15:18.525 INFO:teuthology.orchestra.run.smithi186.stdout: rgw enable usage log = true 2023-10-16T08:15:18.525 INFO:teuthology.orchestra.run.smithi186.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T08:15:18.526 INFO:teuthology.orchestra.run.smithi186.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T08:15:18.526 INFO:teuthology.orchestra.run.smithi186.stdout: client mount timeout = 600 2023-10-16T08:15:18.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug client = 20 2023-10-16T08:15:18.526 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-10-16T08:15:18.526 INFO:teuthology.orchestra.run.smithi186.stdout: pid file = "" 2023-10-16T08:15:18.527 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2023-10-16T08:15:18.527 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2023-10-16T08:15:18.527 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.a] 2023-10-16T08:15:18.527 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.b] 2023-10-16T08:15:18.527 INFO:teuthology.orchestra.run.smithi186.stdout:[mon.c] 2023-10-16T08:15:18.527 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout:[mds] 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds = 20 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout: debug mds balancer = 20 2023-10-16T08:15:18.528 INFO:teuthology.orchestra.run.smithi186.stdout: debug ms = 1 2023-10-16T08:15:18.529 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug frag = True 2023-10-16T08:15:18.529 INFO:teuthology.orchestra.run.smithi186.stdout: mds debug scatterstat = True 2023-10-16T08:15:18.529 INFO:teuthology.orchestra.run.smithi186.stdout: mds op complaint time = 180 2023-10-16T08:15:18.529 INFO:teuthology.orchestra.run.smithi186.stdout: mds verify scatter = True 2023-10-16T08:15:18.529 INFO:teuthology.orchestra.run.smithi186.stdout: osd op complaint time = 180 2023-10-16T08:15:18.529 INFO:teuthology.orchestra.run.smithi186.stdout: rados mon op timeout = 900 2023-10-16T08:15:18.530 INFO:teuthology.orchestra.run.smithi186.stdout: rados osd op timeout = 900 2023-10-16T08:15:18.530 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-16T08:15:18.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-16T08:15:18.606 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-16T08:15:18.607 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:18.607 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-16T08:15:18.616 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:18.617 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-16T08:15:18.673 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:15:18.673 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:18.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T08:15:18.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T08:15:18.754 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:18.754 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:18.805 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:15:18.805 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:18.805 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T08:15:18.806 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T08:15:18.842 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:18.843 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:18.894 INFO:tasks.ceph:Setting up mon nodes... 2023-10-16T08:15:18.894 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-16T08:15:18.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T08:15:18.952 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T08:15:18.957 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T08:15:19.019 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T08:15:19.038 INFO:tasks.ceph:Setting up mds nodes... 2023-10-16T08:15:19.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T08:15:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T08:15:19.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-16T08:15:19.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T08:15:19.221 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T08:15:19.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-16T08:15:19.246 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T08:15:19.307 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T08:15:19.312 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-16T08:15:19.333 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T08:15:19.434 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T08:15:19.438 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-16T08:15:19.460 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-16T08:15:19.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-16T08:15:19.505 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-16T08:15:19.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-16T08:15:19.608 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-16T08:15:19.626 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-16T08:15:19.670 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-16T08:15:19.687 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-16T08:15:19.776 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-16T08:15:19.793 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-16T08:15:19.793 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T08:15:19.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-16T08:15:19.815 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:19.815 INFO:tasks.ceph:role: osd.0 2023-10-16T08:15:19.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:15:19.815 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T08:15:19.876 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:19.877 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:19.877 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:19.877 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:19.878 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:19.878 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:19.900 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-10-16T08:15:19.980 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-10-16T08:15:19.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-16T08:15:20.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-16T08:15:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:20.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:20.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-16T08:15:20.173 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:20.173 INFO:tasks.ceph:role: osd.1 2023-10-16T08:15:20.173 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:15:20.174 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T08:15:20.236 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:20.236 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:20.236 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:20.237 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:20.237 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:20.238 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:20.238 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:20.261 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-10-16T08:15:20.264 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-10-16T08:15:20.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-16T08:15:20.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-16T08:15:20.393 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:20.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:20.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-16T08:15:20.460 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:20.461 INFO:tasks.ceph:role: osd.2 2023-10-16T08:15:20.461 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:15:20.461 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T08:15:20.529 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:20.530 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:20.530 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:20.530 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:20.531 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:20.554 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-10-16T08:15:20.557 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-10-16T08:15:20.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-16T08:15:20.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-16T08:15:20.689 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:20.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:20.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-16T08:15:20.752 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:20.752 INFO:tasks.ceph:role: osd.3 2023-10-16T08:15:20.753 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:15:20.753 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T08:15:20.813 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:20.813 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:20.814 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:20.814 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:20.814 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:20.814 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:20.815 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:20.837 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-10-16T08:15:20.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-10-16T08:15:20.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-16T08:15:20.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-16T08:15:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:20.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:20.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:21.085+0000 7f3a90792640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-16T08:15:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:21.089+0000 7f3a90792640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-16T08:15:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:21.089+0000 7f3a90792640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T08:15:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:21.089+0000 7f3a90792640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T08:15:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:21.089+0000 7f3a90792640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T08:15:21.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:21.089+0000 7f3a90792640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-16T08:15:24.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-16T08:15:24.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:24.193+0000 7f9fe2a7e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-16T08:15:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:24.193+0000 7f9fe2a7e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-16T08:15:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:24.193+0000 7f9fe2a7e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T08:15:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:24.193+0000 7f9fe2a7e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T08:15:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:24.193+0000 7f9fe2a7e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T08:15:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:24.197+0000 7f9fe2a7e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-16T08:15:27.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-16T08:15:27.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:27.297+0000 7fa8ed4a6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-16T08:15:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:27.297+0000 7fa8ed4a6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-16T08:15:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:27.301+0000 7fa8ed4a6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T08:15:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:27.301+0000 7fa8ed4a6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T08:15:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:27.301+0000 7fa8ed4a6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T08:15:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:27.301+0000 7fa8ed4a6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-16T08:15:30.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-16T08:15:30.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:30.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:30.421+0000 7efd97c95640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-16T08:15:30.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:30.421+0000 7efd97c95640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-16T08:15:30.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:30.421+0000 7efd97c95640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T08:15:30.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:30.421+0000 7efd97c95640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T08:15:30.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:30.421+0000 7efd97c95640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T08:15:30.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:30.421+0000 7efd97c95640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-16T08:15:33.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-16T08:15:33.422 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-16T08:15:33.447 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:33.447 INFO:tasks.ceph:role: osd.4 2023-10-16T08:15:33.447 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:15:33.448 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T08:15:33.519 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:33.519 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:33.519 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:33.519 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:33.520 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:33.520 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:33.520 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:33.520 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:33.520 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:33.521 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:33.969 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-10-16T08:15:33.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-10-16T08:15:33.973 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-16T08:15:34.086 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-16T08:15:34.104 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:34.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:34.106 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-16T08:15:34.170 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:34.171 INFO:tasks.ceph:role: osd.5 2023-10-16T08:15:34.171 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:15:34.172 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T08:15:34.247 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:34.247 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:34.248 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:34.248 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:34.248 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:34.248 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:34.248 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:34.248 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:34.249 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:34.249 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:34.736 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-10-16T08:15:34.739 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-10-16T08:15:34.739 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-16T08:15:34.905 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-16T08:15:34.924 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:34.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:34.925 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-16T08:15:34.995 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:34.995 INFO:tasks.ceph:role: osd.6 2023-10-16T08:15:34.995 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:15:34.996 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T08:15:35.066 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:35.066 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:35.066 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:35.066 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:35.067 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:35.067 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:35.067 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:35.067 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:35.067 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:35.068 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:35.547 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-10-16T08:15:35.550 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-10-16T08:15:35.550 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-16T08:15:35.648 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-16T08:15:35.665 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:35.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:35.667 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-16T08:15:35.729 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:15:35.729 INFO:tasks.ceph:role: osd.7 2023-10-16T08:15:35.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:15:35.730 DEBUG:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T08:15:35.801 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:15:35.801 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:15:35.801 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:15:35.801 INFO:teuthology.orchestra.run.smithi186.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:15:35.802 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:15:35.802 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:15:35.802 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:15:35.802 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:15:35.803 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:15:35.803 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:15:36.249 INFO:teuthology.orchestra.run.smithi186.stdout:Discarding blocks...Done. 2023-10-16T08:15:36.252 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2023-10-16T08:15:36.253 DEBUG:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-16T08:15:36.372 DEBUG:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-16T08:15:36.390 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:15:36.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:15:36.392 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:36.519 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:36.518+0000 7fda245b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-16T08:15:36.522 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:36.522+0000 7fda245b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-16T08:15:36.522 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:36.522+0000 7fda245b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T08:15:36.522 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:36.522+0000 7fda245b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T08:15:36.522 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:36.522+0000 7fda245b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T08:15:36.526 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:36.526+0000 7fda245b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-16T08:15:39.521 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-16T08:15:39.543 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:39.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:39.642+0000 7ff86793c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-16T08:15:39.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:39.642+0000 7ff86793c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-16T08:15:39.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:39.642+0000 7ff86793c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T08:15:39.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:39.642+0000 7ff86793c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T08:15:39.644 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:39.642+0000 7ff86793c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T08:15:39.644 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:39.642+0000 7ff86793c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-16T08:15:42.661 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-16T08:15:42.682 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:42.781 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:42.782+0000 7fa7fa228640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-16T08:15:42.782 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:42.782+0000 7fa7fa228640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-16T08:15:42.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:42.782+0000 7fa7fa228640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T08:15:42.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:42.782+0000 7fa7fa228640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T08:15:42.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:42.782+0000 7fa7fa228640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T08:15:42.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:42.782+0000 7fa7fa228640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-16T08:15:45.780 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-16T08:15:45.801 DEBUG:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:45.898 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:45.898+0000 7f0460dad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-16T08:15:45.899 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:45.898+0000 7f0460dad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-16T08:15:45.900 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:45.898+0000 7f0460dad640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T08:15:45.900 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:45.898+0000 7f0460dad640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T08:15:45.900 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:45.898+0000 7f0460dad640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T08:15:45.900 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:45.898+0000 7f0460dad640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-16T08:15:46.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:15:46.516 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:15:48.895 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-16T08:15:48.916 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-16T08:15:48.917 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:48.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-16T08:15:48.940 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:48.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-16T08:15:49.006 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-16T08:15:49.070 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-16T08:15:49.134 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-16T08:15:49.198 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-16T08:15:49.262 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-16T08:15:49.326 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.326 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-16T08:15:49.347 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.348 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-16T08:15:49.409 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.409 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-16T08:15:49.473 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.473 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-16T08:15:49.540 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.541 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-16T08:15:49.600 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.601 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-16T08:15:49.665 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.665 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-16T08:15:49.728 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.729 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-16T08:15:49.738 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:49.739 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-16T08:15:49.789 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.789 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-16T08:15:49.799 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:49.799 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-16T08:15:49.849 INFO:tasks.ceph:Adding keys to all mons... 2023-10-16T08:15:49.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T08:15:49.854 DEBUG:teuthology.orchestra.run.smithi186:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T08:15:49.897 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2023-10-16T08:15:49.897 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAW8SxlPtfFOBAAOXlHIg/7e75c4MipG5PHGw== 2023-10-16T08:15:49.898 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-10-16T08:15:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8Sxlq7jUBRAAZ/PoSPxeTxzGVyZoXvddGA== 2023-10-16T08:15:49.898 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-10-16T08:15:49.898 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlZ8QwDRAAa6bkfiFmL7N90yyiFUBOsg== 2023-10-16T08:15:49.898 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2023-10-16T08:15:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAZ8SxltyuHBRAA523N7XKVdNMwSLQo24Vvvg== 2023-10-16T08:15:49.899 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2023-10-16T08:15:49.899 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAc8Sxl88vjCxAAh5b4FThP+7Y7k0nYLylx5Q== 2023-10-16T08:15:49.899 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2023-10-16T08:15:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAf8SxlfcEoEhAABJMIQKe43NGUqMCXCq6HEA== 2023-10-16T08:15:49.900 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2023-10-16T08:15:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAi8SxlomRcGRAAIu96ZzCy18pSgsR3hD/Y7Q== 2023-10-16T08:15:49.900 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2023-10-16T08:15:49.900 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlT7Q7ARAAVxrt9krDiqnEkQ0+I4duzQ== 2023-10-16T08:15:49.900 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-10-16T08:15:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlD5VrEhAAoIJTZvkt2JVMQDjd9QNNnA== 2023-10-16T08:15:49.901 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-10-16T08:15:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxliSn5GRAA8atnMBpEoXUyjpqQtoGWgw== 2023-10-16T08:15:49.901 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2023-10-16T08:15:49.901 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAo8SxlXxEEHxAAqohCELe6In6sSjbBTHY+OQ== 2023-10-16T08:15:49.901 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2023-10-16T08:15:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAr8SxloVNhJhAADF2erI7/queuEM5kSqqJkQ== 2023-10-16T08:15:49.902 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2023-10-16T08:15:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAu8SxlRtStLhAAIzAuqOD9H7iiabi1P67nNA== 2023-10-16T08:15:49.902 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2023-10-16T08:15:49.902 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAx8SxlthuoNRAAubSeLAllgyLqaRY4z4G35w== 2023-10-16T08:15:49.903 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:15:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:15:49.903 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2023-10-16T08:15:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlBylLJBAAXNMyfzVb5MDsSdSxVN59lg== 2023-10-16T08:15:49.903 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2023-10-16T08:15:49.903 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlA8EAKBAA7lsMEVeNC7t3xvgCLWoYcQ== 2023-10-16T08:15:49.904 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2023-10-16T08:15:49.904 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8Sxl1dxZLhAAqhBqKEmTcorRaKo5YmmzvA== 2023-10-16T08:15:49.910 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.y] 2023-10-16T08:15:49.911 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAW8SxlPtfFOBAAOXlHIg/7e75c4MipG5PHGw== 2023-10-16T08:15:49.911 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.a] 2023-10-16T08:15:49.911 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8Sxlq7jUBRAAZ/PoSPxeTxzGVyZoXvddGA== 2023-10-16T08:15:49.911 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.c] 2023-10-16T08:15:49.911 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxlZ8QwDRAAa6bkfiFmL7N90yyiFUBOsg== 2023-10-16T08:15:49.912 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.0] 2023-10-16T08:15:49.912 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAZ8SxltyuHBRAA523N7XKVdNMwSLQo24Vvvg== 2023-10-16T08:15:49.912 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.1] 2023-10-16T08:15:49.912 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAc8Sxl88vjCxAAh5b4FThP+7Y7k0nYLylx5Q== 2023-10-16T08:15:49.912 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.2] 2023-10-16T08:15:49.912 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAf8SxlfcEoEhAABJMIQKe43NGUqMCXCq6HEA== 2023-10-16T08:15:49.913 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.3] 2023-10-16T08:15:49.913 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAi8SxlomRcGRAAIu96ZzCy18pSgsR3hD/Y7Q== 2023-10-16T08:15:49.913 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.x] 2023-10-16T08:15:49.913 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxlT7Q7ARAAVxrt9krDiqnEkQ0+I4duzQ== 2023-10-16T08:15:49.913 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.b] 2023-10-16T08:15:49.914 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxlD5VrEhAAoIJTZvkt2JVMQDjd9QNNnA== 2023-10-16T08:15:49.914 INFO:teuthology.orchestra.run.smithi186.stdout:[mds.d] 2023-10-16T08:15:49.914 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxliSn5GRAA8atnMBpEoXUyjpqQtoGWgw== 2023-10-16T08:15:49.914 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.4] 2023-10-16T08:15:49.914 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAo8SxlXxEEHxAAqohCELe6In6sSjbBTHY+OQ== 2023-10-16T08:15:49.915 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.5] 2023-10-16T08:15:49.915 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAr8SxloVNhJhAADF2erI7/queuEM5kSqqJkQ== 2023-10-16T08:15:49.915 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.6] 2023-10-16T08:15:49.915 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAu8SxlRtStLhAAIzAuqOD9H7iiabi1P67nNA== 2023-10-16T08:15:49.915 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.7] 2023-10-16T08:15:49.915 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAx8SxlthuoNRAAubSeLAllgyLqaRY4z4G35w== 2023-10-16T08:15:49.916 INFO:teuthology.orchestra.run.smithi186.stdout:[client.0] 2023-10-16T08:15:49.916 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:15:49.916 INFO:teuthology.orchestra.run.smithi186.stdout:[client.2] 2023-10-16T08:15:49.916 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxlBylLJBAAXNMyfzVb5MDsSdSxVN59lg== 2023-10-16T08:15:49.917 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2023-10-16T08:15:49.917 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8SxlA8EAKBAA7lsMEVeNC7t3xvgCLWoYcQ== 2023-10-16T08:15:49.917 INFO:teuthology.orchestra.run.smithi186.stdout:[client.3] 2023-10-16T08:15:49.917 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQAX8Sxl1dxZLhAAqhBqKEmTcorRaKo5YmmzvA== 2023-10-16T08:15:49.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:15:49.951 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:15:50.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.019 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.082 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.143 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.201 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.260 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.319 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:15:50.378 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:15:50.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.435 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.497 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:15:50.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.560 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.619 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.678 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.737 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:15:50.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.796 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.858 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.920 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:50.982 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:15:51.040 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-16T08:15:51.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-16T08:15:51.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T08:15:51.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-16T08:15:51.200 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-16T08:15:51.220 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T08:15:51.331 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-16T08:15:51.352 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-16T08:15:51.412 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T08:15:51.525 DEBUG:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-16T08:15:51.546 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:51.550 DEBUG:teuthology.orchestra.run.smithi186:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:15:51.596 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-16T08:15:51.597 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-16T08:15:51.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-16T08:15:51.603 INFO:tasks.ceph.mon.a:Started 2023-10-16T08:15:51.603 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-16T08:15:51.604 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-16T08:15:51.640 INFO:tasks.ceph.mon.b:Started 2023-10-16T08:15:51.640 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-16T08:15:51.641 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-16T08:15:51.644 INFO:tasks.ceph.mon.c:Started 2023-10-16T08:15:51.644 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-16T08:15:51.645 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T08:15:51.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T08:15:51.649 INFO:tasks.ceph.mgr.y:Started 2023-10-16T08:15:51.650 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-16T08:15:51.650 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-16T08:15:51.654 INFO:tasks.ceph.mgr.x:Started 2023-10-16T08:15:51.654 INFO:tasks.ceph:Setting crush tunables to default 2023-10-16T08:15:51.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2023-10-16T08:15:51.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.796+0000 7f59d92d3640 1 Processor -- start 2023-10-16T08:15:51.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.796+0000 7f59d92d3640 1 -- start start 2023-10-16T08:15:51.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d92d3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:51.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d92d3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:51.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d92d3640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d40cfb30 con 0x7f59d40d3320 2023-10-16T08:15:51.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d92d3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d40d0880 con 0x7f59d415cdc0 2023-10-16T08:15:51.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d92d3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40b1c00 con 0x7f59d40cf210 2023-10-16T08:15:51.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:51.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58262/0 (socket says 172.21.15.32:58262) 2023-10-16T08:15:51.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 learned_addr learned my addr 172.21.15.32:0/1643037445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:15:51.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:51.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:51.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:51.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T08:15:51.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:51.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-16T08:15:51.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:51.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:51.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:51.800+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T08:15:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-16T08:15:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T08:15:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.004+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T08:15:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-16T08:15:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T08:15:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:52.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:52.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:52.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:52.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:52.408+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T08:15:53.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T08:15:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-16T08:15:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:53.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:53.212+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T08:15:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 msgr2=0x7f59d40cf5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:15:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59d40cf210 0x7f59d40cf5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 msgr2=0x7f59d40b2ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:15:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.32:3300/0 conn(0x7f59d415cdc0 0x7f59d40b2ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 -- 172.21.15.32:0/1643037445 >> v1:172.21.15.186:6789/0 conn(0x7f59d40d3320 legacy=0x7f59d40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 -- 172.21.15.32:0/1643037445 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d40b1ee0 con 0x7f59c00036d0 2023-10-16T08:15:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d1ffb640 1 -- 172.21.15.32:0/1643037445 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d40d0880 con 0x7f59c0007360 2023-10-16T08:15:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:54.800+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T08:15:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.004+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.004+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.004+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.004+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:55.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.004+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T08:15:55.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.408+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:55.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.408+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:55.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.408+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:55.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.408+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:55.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:55.408+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T08:15:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.212+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.212+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:15:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.212+0000 7f59d37fe640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:15:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.212+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:15:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.212+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T08:15:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106270476 0 0) 0x7f59d40b1ee0 con 0x7f59c00036d0 2023-10-16T08:15:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59bc003530 con 0x7f59c00036d0 2023-10-16T08:15:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2389958219 0 0) 0x7f59bc003530 con 0x7f59c00036d0 2023-10-16T08:15:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 msgr2=0x7f59c0009820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:15:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 --2- 172.21.15.32:0/1643037445 >> v2:172.21.15.186:3300/0 conn(0x7f59c0007360 0x7f59c0009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c000af80 con 0x7f59c00036d0 2023-10-16T08:15:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f59c400d5d0 con 0x7f59c00036d0 2023-10-16T08:15:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 >> v1:172.21.15.32:6789/0 conn(0x7f59c00036d0 legacy=0x7f59c0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d8ad2640 1 -- 172.21.15.32:0/1643037445 reap_dead start 2023-10-16T08:15:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc003980 0x7f59bc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc0052b0 0x7f59bc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40d0880 con 0x7f59bc003980 2023-10-16T08:15:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00c200 con 0x7f59bc0052b0 2023-10-16T08:15:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.952+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00c3c0 con 0x7f59bc0047d0 2023-10-16T08:15:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc003980 0x7f59bc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d8ad2640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc0052b0 0x7f59bc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d37fe640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 msgr2=0x7f59bc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:56.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59bc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc0052b0 msgr2=0x7f59bc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc0052b0 0x7f59bc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 -- 172.21.15.32:0/1643037445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc003530 con 0x7f59bc003980 2023-10-16T08:15:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.956+0000 7f59d3fff640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc003980 0x7f59bc003d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006fd0 tx=0x7f59c401da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29607ae4d4c7214a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40059a0 con 0x7f59bc003980 2023-10-16T08:15:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40aa070 con 0x7f59bc003980 2023-10-16T08:15:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d2ffd640 1 -- 172.21.15.32:0/1643037445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4005d60 con 0x7f59bc003980 2023-10-16T08:15:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 -- 172.21.15.32:0/1643037445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc003980 msgr2=0x7f59bc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc003980 0x7f59bc003d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006fd0 tx=0x7f59c401da50 comp rx=0 tx=0).stop 2023-10-16T08:15:56.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 -- 172.21.15.32:0/1643037445 shutdown_connections 2023-10-16T08:15:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc0052b0 0x7f59bc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59bc004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 --2- 172.21.15.32:0/1643037445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc003980 0x7f59bc003d80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 -- 172.21.15.32:0/1643037445 >> 172.21.15.32:0/1643037445 conn(0x7f59d40d2800 msgr2=0x7f59d40d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:56.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 -- 172.21.15.32:0/1643037445 shutdown_connections 2023-10-16T08:15:56.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 -- 172.21.15.32:0/1643037445 wait complete. 2023-10-16T08:15:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 Processor -- start 2023-10-16T08:15:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.960+0000 7f59d92d3640 1 -- start start 2023-10-16T08:15:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc003980 0x7f59d41f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59d40b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 0x7f59d415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4155520 con 0x7f59bc0052b0 2023-10-16T08:15:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40b7b90 con 0x7f59bc003980 2023-10-16T08:15:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41f6330 con 0x7f59bc0047d0 2023-10-16T08:15:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 0x7f59d415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc003980 0x7f59d41f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 0x7f59d415d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40850/0 (socket says 172.21.15.32:40850) 2023-10-16T08:15:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 -- 172.21.15.32:0/3213749106 learned_addr learned my addr 172.21.15.32:0/3213749106 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:15:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d37fe640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59d40b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 -- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 msgr2=0x7f59d40b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59d40b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 -- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc003980 msgr2=0x7f59d41f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc003980 0x7f59d41f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 -- 172.21.15.32:0/3213749106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40b6890 con 0x7f59bc0052b0 2023-10-16T08:15:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d8ad2640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 0x7f59d415d1a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009de0 tx=0x7f59cc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00f780 con 0x7f59bc0052b0 2023-10-16T08:15:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00f8e0 con 0x7f59bc0052b0 2023-10-16T08:15:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00fc90 con 0x7f59bc0052b0 2023-10-16T08:15:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40b6b70 con 0x7f59bc0052b0 2023-10-16T08:15:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d40b7130 con 0x7f59bc0052b0 2023-10-16T08:15:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f59cc01e760 con 0x7f59bc0052b0 2023-10-16T08:15:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f59cc0134d0 con 0x7f59bc0052b0 2023-10-16T08:15:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.964+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d40b7350 con 0x7f59bc0052b0 2023-10-16T08:15:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:56.968+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f59cc01eba0 con 0x7f59bc0052b0 2023-10-16T08:15:57.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.032+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f59d404e9c0 con 0x7f59bc0052b0 2023-10-16T08:15:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d0ff9640 1 -- 172.21.15.32:0/3213749106 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f59cc01ed40 con 0x7f59bc0052b0 2023-10-16T08:15:57.038 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2023-10-16T08:15:57.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 msgr2=0x7f59d415d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 0x7f59d415d1a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009de0 tx=0x7f59cc007700 comp rx=0 tx=0).stop 2023-10-16T08:15:57.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 shutdown_connections 2023-10-16T08:15:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59bc0052b0 0x7f59d415d1a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f59bc0047d0 0x7f59d40b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 --2- 172.21.15.32:0/3213749106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59bc003980 0x7f59d41f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 >> 172.21.15.32:0/3213749106 conn(0x7f59d40d2800 msgr2=0x7f59d40b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 shutdown_connections 2023-10-16T08:15:57.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:57.036+0000 7f59d92d3640 1 -- 172.21.15.32:0/3213749106 wait complete. 2023-10-16T08:15:57.059 INFO:tasks.ceph:check_enable_crimson: False 2023-10-16T08:15:57.059 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-16T08:15:57.060 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:57.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-16T08:15:57.080 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:57.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-16T08:15:57.144 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:57.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-16T08:15:57.192 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:57.190+0000 7f4fc5d63180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T08:15:57.205 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:15:57.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-16T08:15:57.268 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:57.269 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-16T08:15:57.289 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:57.290 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-16T08:15:57.353 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:57.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-16T08:15:57.417 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:15:57.417 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-16T08:15:57.481 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 4a1b4a42-2706-464f-a2b4-b23c887af4ff 0 2023-10-16T08:15:57.661 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.658+0000 7f4ccc2c9640 1 Processor -- start 2023-10-16T08:15:57.661 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.658+0000 7f4ccc2c9640 1 -- start start 2023-10-16T08:15:57.663 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4ccc2c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cc40cdf50 0x7f4cc40b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.663 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4ccc2c9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc415d010 con 0x7f4cc40d1fa0 2023-10-16T08:15:57.663 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4ccc2c9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc40b74d0 con 0x7f4cc415cc30 2023-10-16T08:15:57.664 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4ccc2c9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc40b77b0 con 0x7f4cc40cdf50 2023-10-16T08:15:57.664 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f4cc415cc30 0x7f4cc40cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:47150/0 (socket says 172.21.15.186:47150) 2023-10-16T08:15:57.664 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 -- 172.21.15.186:0/1959920290 learned_addr learned my addr 172.21.15.186:0/1959920290 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-10-16T08:15:57.664 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca866640 1 --2- 172.21.15.186:0/1959920290 >> v2:172.21.15.32:3300/0 conn(0x7f4cc40cdf50 0x7f4cc40b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.664 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348941788 0 0) 0x7f4cc40b74d0 con 0x7f4cc415cc30 2023-10-16T08:15:57.665 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb0003530 con 0x7f4cc415cc30 2023-10-16T08:15:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437267957 0 0) 0x7f4cb0003530 con 0x7f4cc415cc30 2023-10-16T08:15:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 >> v2:172.21.15.32:3300/0 conn(0x7f4cc40cdf50 msgr2=0x7f4cc40b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 --2- 172.21.15.186:0/1959920290 >> v2:172.21.15.32:3300/0 conn(0x7f4cc40cdf50 0x7f4cc40b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.667 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 >> v1:172.21.15.32:6789/0 conn(0x7f4cc40d1fa0 legacy=0x7f4cc40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc40b7a90 con 0x7f4cc415cc30 2023-10-16T08:15:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f4cc0001680 con 0x7f4cc415cc30 2023-10-16T08:15:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca866640 1 --2- 172.21.15.186:0/1959920290 >> v2:172.21.15.32:3300/0 conn(0x7f4cc40cdf50 0x7f4cc40b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 >> v1:172.21.15.186:6789/0 conn(0x7f4cc415cc30 legacy=0x7f4cc40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.668 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cb0003980 0x7f4cb0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.669 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cb0004380 0x7f4cb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.669 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0005050 0x7f4cb000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.669 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40b77b0 con 0x7f4cb0005050 2023-10-16T08:15:57.669 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb000bf00 con 0x7f4cb0004380 2023-10-16T08:15:57.669 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb000c0c0 con 0x7f4cb0003980 2023-10-16T08:15:57.670 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cb0003980 0x7f4cb0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.670 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cc9864640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cb0004380 0x7f4cb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.670 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca866640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0005050 0x7f4cb000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.670 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 -- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cb0004380 msgr2=0x7f4cb0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:57.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cb0004380 0x7f4cb0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 -- 172.21.15.186:0/1959920290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0005050 msgr2=0x7f4cb000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:57.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0005050 0x7f4cb000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca065640 1 -- 172.21.15.186:0/1959920290 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb0003530 con 0x7f4cb0003980 2023-10-16T08:15:57.671 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca866640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0005050 0x7f4cb000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:57.672 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.662+0000 7f4cca866640 1 -- 172.21.15.186:0/1959920290 reap_dead start 2023-10-16T08:15:57.675 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4cca065640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cb0003980 0x7f4cb0003d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c210 tx=0x7f4cc001cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaf7cf13ba9b92b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:57.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0005ae0 con 0x7f4cb0003980 2023-10-16T08:15:57.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc0005c60 con 0x7f4cb0003980 2023-10-16T08:15:57.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4cc9063640 1 -- 172.21.15.186:0/1959920290 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00a5b20 con 0x7f4cb0003980 2023-10-16T08:15:57.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cb0003980 msgr2=0x7f4cb0003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.676 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cb0003980 0x7f4cb0003d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c210 tx=0x7f4cc001cbe0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1959920290 shutdown_connections 2023-10-16T08:15:57.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 --2- 172.21.15.186:0/1959920290 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cb0003980 0x7f4cb0003d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1959920290 >> 172.21.15.186:0/1959920290 conn(0x7f4cc406b750 msgr2=0x7f4cc406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:57.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1959920290 shutdown_connections 2023-10-16T08:15:57.677 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.666+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1959920290 wait complete. 2023-10-16T08:15:57.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.670+0000 7f4ccc2c9640 1 Processor -- start 2023-10-16T08:15:57.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.670+0000 7f4ccc2c9640 1 -- start start 2023-10-16T08:15:57.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.670+0000 7f4ccc2c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0003980 0x7f4cc415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.678 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.670+0000 7f4ccc2c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 0x7f4cc414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 0x7f4cc415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc414f690 con 0x7f4cb0003980 2023-10-16T08:15:57.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40c9550 con 0x7f4cc40cc380 2023-10-16T08:15:57.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40c8980 con 0x7f4cc41595b0 2023-10-16T08:15:57.679 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cca866640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 0x7f4cc415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 0x7f4cc414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cca866640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 0x7f4cc415aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:34468/0 (socket says 172.21.15.186:34468) 2023-10-16T08:15:57.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cca866640 1 -- 172.21.15.186:0/1483529630 learned_addr learned my addr 172.21.15.186:0/1483529630 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:57.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 0x7f4cc414f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34576/0 (socket says 172.21.15.186:34576) 2023-10-16T08:15:57.680 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cca065640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0003980 0x7f4cc415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.681 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 -- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 msgr2=0x7f4cc415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.681 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 0x7f4cc415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.681 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 -- 172.21.15.186:0/1483529630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0003980 msgr2=0x7f4cc415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.681 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0003980 0x7f4cc415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.681 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 -- 172.21.15.186:0/1483529630 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc40c8c20 con 0x7f4cc41595b0 2023-10-16T08:15:57.682 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cca866640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 0x7f4cc415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:57.682 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cc9864640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 0x7f4cc414f150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4007fd0 tx=0x7f4cb400fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:57.682 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40050d0 con 0x7f4cc41595b0 2023-10-16T08:15:57.682 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4005250 con 0x7f4cc41595b0 2023-10-16T08:15:57.683 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc40c8f00 con 0x7f4cc41595b0 2023-10-16T08:15:57.683 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400d650 con 0x7f4cc41595b0 2023-10-16T08:15:57.683 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc40cc800 con 0x7f4cc41595b0 2023-10-16T08:15:57.683 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4cb400dab0 con 0x7f4cc41595b0 2023-10-16T08:15:57.683 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4cb4003d10 con 0x7f4cc41595b0 2023-10-16T08:15:57.684 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.674+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc40c9c20 con 0x7f4cc41595b0 2023-10-16T08:15:57.684 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.678+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4cb4003600 con 0x7f4cc41595b0 2023-10-16T08:15:57.744 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.742+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4a1b4a42-2706-464f-a2b4-b23c887af4ff", "id": 0} v 0) v1 -- 0x7f4cc4067230 con 0x7f4cc41595b0 2023-10-16T08:15:57.751 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4cbaffd640 1 -- 172.21.15.186:0/1483529630 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a1b4a42-2706-464f-a2b4-b23c887af4ff", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4cb40037a0 con 0x7f4cc41595b0 2023-10-16T08:15:57.752 INFO:teuthology.orchestra.run.smithi186.stdout:0 2023-10-16T08:15:57.753 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 msgr2=0x7f4cc414f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.753 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 0x7f4cc414f150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4007fd0 tx=0x7f4cb400fb40 comp rx=0 tx=0).stop 2023-10-16T08:15:57.753 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 shutdown_connections 2023-10-16T08:15:57.753 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc40cc380 0x7f4cc415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.754 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc41595b0 0x7f4cc414f150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.754 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 --2- 172.21.15.186:0/1483529630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0003980 0x7f4cc415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.754 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 >> 172.21.15.186:0/1483529630 conn(0x7f4cc406b750 msgr2=0x7f4cc40d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:57.755 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 shutdown_connections 2023-10-16T08:15:57.755 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.750+0000 7f4ccc2c9640 1 -- 172.21.15.186:0/1483529630 wait complete. 2023-10-16T08:15:57.773 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new f773ecc7-b793-44c4-ae48-592c08df1272 1 2023-10-16T08:15:57.777 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:57.774+0000 7f4fc5d63180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T08:15:57.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.906+0000 7ff9ba572640 1 Processor -- start 2023-10-16T08:15:57.909 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.906+0000 7ff9ba572640 1 -- start start 2023-10-16T08:15:57.910 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9ba572640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9b415cc30 0x7ff9b40b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9ba572640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff9b40d1fa0 0x7ff9b40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9ba572640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b415d010 con 0x7ff9b40cd840 2023-10-16T08:15:57.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9ba572640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40b74d0 con 0x7ff9b415cc30 2023-10-16T08:15:57.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9ba572640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40b77b0 con 0x7ff9b40d1fa0 2023-10-16T08:15:57.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b37fe640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ff9b40cd840 0x7ff9b40cdc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:47164/0 (socket says 172.21.15.186:47164) 2023-10-16T08:15:57.911 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b37fe640 1 -- 172.21.15.186:0/4076270270 learned_addr learned my addr 172.21.15.186:0/4076270270 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-10-16T08:15:57.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b8b0f640 1 --2- 172.21.15.186:0/4076270270 >> v2:172.21.15.186:3300/0 conn(0x7ff9b40d1fa0 0x7ff9b40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b3fff640 1 --2- 172.21.15.186:0/4076270270 >> v2:172.21.15.32:3300/0 conn(0x7ff9b415cc30 0x7ff9b40b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.912 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b8b0f640 1 -- 172.21.15.186:0/4076270270 >> v2:172.21.15.32:3300/0 conn(0x7ff9b415cc30 msgr2=0x7ff9b40b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:57.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b8b0f640 1 --2- 172.21.15.186:0/4076270270 >> v2:172.21.15.32:3300/0 conn(0x7ff9b415cc30 0x7ff9b40b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b8b0f640 1 -- 172.21.15.186:0/4076270270 >> v1:172.21.15.186:6789/0 conn(0x7ff9b40cd840 legacy=0x7ff9b40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b8b0f640 1 -- 172.21.15.186:0/4076270270 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40b7a90 con 0x7ff9b40d1fa0 2023-10-16T08:15:57.913 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.910+0000 7ff9b2ffd640 1 -- 172.21.15.186:0/4076270270 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665615846 0 0) 0x7ff9b415d010 con 0x7ff9b40cd840 2023-10-16T08:15:57.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9b8b0f640 1 --2- 172.21.15.186:0/4076270270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40d1fa0 0x7ff9b40b5be0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9b408be20 tx=0x7ff9ac01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b0534e2d9bb090 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:57.914 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9b2ffd640 1 -- 172.21.15.186:0/4076270270 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0a59c0 con 0x7ff9b40d1fa0 2023-10-16T08:15:57.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9b2ffd640 1 -- 172.21.15.186:0/4076270270 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac0a5b20 con 0x7ff9b40d1fa0 2023-10-16T08:15:57.915 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9b2ffd640 1 -- 172.21.15.186:0/4076270270 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0ad440 con 0x7ff9b40d1fa0 2023-10-16T08:15:57.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 -- 172.21.15.186:0/4076270270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40d1fa0 msgr2=0x7ff9b40b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 --2- 172.21.15.186:0/4076270270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40d1fa0 0x7ff9b40b5be0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9b408be20 tx=0x7ff9ac01c8e0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 -- 172.21.15.186:0/4076270270 shutdown_connections 2023-10-16T08:15:57.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 --2- 172.21.15.186:0/4076270270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40d1fa0 0x7ff9b40b5be0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.916 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 --2- 172.21.15.186:0/4076270270 >> v2:172.21.15.32:3300/0 conn(0x7ff9b415cc30 0x7ff9b40b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 -- 172.21.15.186:0/4076270270 >> 172.21.15.186:0/4076270270 conn(0x7ff9b406b750 msgr2=0x7ff9b406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:57.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 -- 172.21.15.186:0/4076270270 shutdown_connections 2023-10-16T08:15:57.917 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 -- 172.21.15.186:0/4076270270 wait complete. 2023-10-16T08:15:57.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 Processor -- start 2023-10-16T08:15:57.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.914+0000 7ff9ba572640 1 -- start start 2023-10-16T08:15:57.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 0x7ff9b4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9b40d1fa0 0x7ff9b4158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.920 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9b415cc30 0x7ff9b40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:57.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40cb470 con 0x7ff9b415cc30 2023-10-16T08:15:57.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40cb890 con 0x7ff9b40cd840 2023-10-16T08:15:57.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40ca7f0 con 0x7ff9b40d1fa0 2023-10-16T08:15:57.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 0x7ff9b4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.921 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 0x7ff9b4157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:34500/0 (socket says 172.21.15.186:34500) 2023-10-16T08:15:57.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 -- 172.21.15.186:0/967589818 learned_addr learned my addr 172.21.15.186:0/967589818 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:57.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b37fe640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9b40d1fa0 0x7ff9b4158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b8b0f640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9b415cc30 0x7ff9b40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:57.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 -- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9b40d1fa0 msgr2=0x7ff9b4158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.922 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9b40d1fa0 0x7ff9b4158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 -- 172.21.15.186:0/967589818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9b415cc30 msgr2=0x7ff9b40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9b415cc30 0x7ff9b40cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:57.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 -- 172.21.15.186:0/967589818 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac009a70 con 0x7ff9b40cd840 2023-10-16T08:15:57.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b3fff640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 0x7ff9b4157ff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400e870 tx=0x7ff9a400ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:57.923 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4009510 con 0x7ff9b40cd840 2023-10-16T08:15:57.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400eec0 con 0x7ff9b40cd840 2023-10-16T08:15:57.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400cb40 con 0x7ff9b40cd840 2023-10-16T08:15:57.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40caad0 con 0x7ff9b40cd840 2023-10-16T08:15:57.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff9a4013470 con 0x7ff9b40cd840 2023-10-16T08:15:57.924 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b40cb090 con 0x7ff9b40cd840 2023-10-16T08:15:57.925 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff9a4013cc0 con 0x7ff9b40cd840 2023-10-16T08:15:57.925 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.918+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b40cb6e0 con 0x7ff9b40cd840 2023-10-16T08:15:57.925 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.922+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff9a403bb40 con 0x7ff9b40cd840 2023-10-16T08:15:57.982 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:57.982+0000 7f4fc5d63180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T08:15:57.984 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:57.980+0000 7ff82880c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T08:15:57.992 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.990+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f773ecc7-b793-44c4-ae48-592c08df1272", "id": 1} v 0) v1 -- 0x7ff9b40cb220 con 0x7ff9b40cd840 2023-10-16T08:15:57.998 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9b0ff9640 1 -- 172.21.15.186:0/967589818 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f773ecc7-b793-44c4-ae48-592c08df1272", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff9a402a070 con 0x7ff9b40cd840 2023-10-16T08:15:57.998 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-10-16T08:15:57.999 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 msgr2=0x7ff9b4157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:57.999 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 0x7ff9b4157ff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400e870 tx=0x7ff9a400ec70 comp rx=0 tx=0).stop 2023-10-16T08:15:58.000 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 shutdown_connections 2023-10-16T08:15:58.000 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9b415cc30 0x7ff9b40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.000 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff9b40d1fa0 0x7ff9b4158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.000 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 --2- 172.21.15.186:0/967589818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9b40cd840 0x7ff9b4157ff0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.001 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 >> 172.21.15.186:0/967589818 conn(0x7ff9b406b750 msgr2=0x7ff9b40728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.001 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 shutdown_connections 2023-10-16T08:15:58.001 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:57.998+0000 7ff9ba572640 1 -- 172.21.15.186:0/967589818 wait complete. 2023-10-16T08:15:58.018 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 68eab466-4b6b-406f-86f7-6414a3c00a32 2 2023-10-16T08:15:58.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 Processor -- start 2023-10-16T08:15:58.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 -- start start 2023-10-16T08:15:58.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e680cf210 0x7f2e680cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 0x7f2e680b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e680cfb30 con 0x7f2e680d3320 2023-10-16T08:15:58.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e680d0880 con 0x7f2e680cf210 2023-10-16T08:15:58.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6dba7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680b1c00 con 0x7f2e6815cdc0 2023-10-16T08:15:58.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 0x7f2e680b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 0x7f2e680b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34600/0 (socket says 172.21.15.186:34600) 2023-10-16T08:15:58.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e67fff640 1 -- 172.21.15.186:0/4184491411 learned_addr learned my addr 172.21.15.186:0/4184491411 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:58.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.154+0000 7f2e6cba5640 1 --2- 172.21.15.186:0/4184491411 >> v2:172.21.15.32:3300/0 conn(0x7f2e680cf210 0x7f2e680cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e67fff640 1 -- 172.21.15.186:0/4184491411 >> v2:172.21.15.32:3300/0 conn(0x7f2e680cf210 msgr2=0x7f2e680cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:58.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e67fff640 1 --2- 172.21.15.186:0/4184491411 >> v2:172.21.15.32:3300/0 conn(0x7f2e680cf210 0x7f2e680cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e67fff640 1 -- 172.21.15.186:0/4184491411 >> v1:172.21.15.32:6789/0 conn(0x7f2e680d3320 legacy=0x7f2e680b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e67fff640 1 -- 172.21.15.186:0/4184491411 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e680b1ee0 con 0x7f2e6815cdc0 2023-10-16T08:15:58.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e67fff640 1 --2- 172.21.15.186:0/4184491411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 0x7f2e680b2ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009870 tx=0x7f2e680b4100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8438686b15e702a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e677fe640 1 -- 172.21.15.186:0/4184491411 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e540aa670 con 0x7f2e6815cdc0 2023-10-16T08:15:58.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e677fe640 1 -- 172.21.15.186:0/4184491411 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e540aa7d0 con 0x7f2e6815cdc0 2023-10-16T08:15:58.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e677fe640 1 -- 172.21.15.186:0/4184491411 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e540aab60 con 0x7f2e6815cdc0 2023-10-16T08:15:58.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4184491411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 msgr2=0x7f2e680b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4184491411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 0x7f2e680b2ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009870 tx=0x7f2e680b4100 comp rx=0 tx=0).stop 2023-10-16T08:15:58.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4184491411 shutdown_connections 2023-10-16T08:15:58.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4184491411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e6815cdc0 0x7f2e680b2ca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4184491411 >> v2:172.21.15.32:3300/0 conn(0x7f2e680cf210 0x7f2e680cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.158+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4184491411 >> 172.21.15.186:0/4184491411 conn(0x7f2e680d2800 msgr2=0x7f2e680d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4184491411 shutdown_connections 2023-10-16T08:15:58.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4184491411 wait complete. 2023-10-16T08:15:58.165 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 Processor -- start 2023-10-16T08:15:58.166 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 -- start start 2023-10-16T08:15:58.166 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 0x7f2e6815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.166 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e680d3320 0x7f2e681f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.166 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6815cdc0 0x7f2e681f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.166 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680cb260 con 0x7f2e6815cdc0 2023-10-16T08:15:58.167 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680cb6f0 con 0x7f2e680d3320 2023-10-16T08:15:58.167 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6dba7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680ca650 con 0x7f2e680cf210 2023-10-16T08:15:58.167 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e680d3320 0x7f2e681f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.167 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 0x7f2e6815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.167 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 0x7f2e6815af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34602/0 (socket says 172.21.15.186:34602) 2023-10-16T08:15:58.168 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 -- 172.21.15.186:0/4042318050 learned_addr learned my addr 172.21.15.186:0/4042318050 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:58.168 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6d3a6640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6815cdc0 0x7f2e681f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.168 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 -- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e680d3320 msgr2=0x7f2e681f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.168 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e680d3320 0x7f2e681f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.168 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 -- 172.21.15.186:0/4042318050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6815cdc0 msgr2=0x7f2e681f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6815cdc0 0x7f2e681f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 -- 172.21.15.186:0/4042318050 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e540094d0 con 0x7f2e680cf210 2023-10-16T08:15:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e67fff640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e680d3320 0x7f2e681f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.162+0000 7f2e6cba5640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 0x7f2e6815af40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00e930 tx=0x7f2e5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c009690 con 0x7f2e680cf210 2023-10-16T08:15:58.170 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00ca00 con 0x7f2e680cf210 2023-10-16T08:15:58.170 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e680ca930 con 0x7f2e680cf210 2023-10-16T08:15:58.170 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00cd90 con 0x7f2e680cf210 2023-10-16T08:15:58.170 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e680caef0 con 0x7f2e680cf210 2023-10-16T08:15:58.170 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2e5c013770 con 0x7f2e680cf210 2023-10-16T08:15:58.171 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f2e5c013e40 con 0x7f2e680cf210 2023-10-16T08:15:58.171 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6804e9c0 con 0x7f2e680cf210 2023-10-16T08:15:58.172 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.166+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2e5c03bb40 con 0x7f2e680cf210 2023-10-16T08:15:58.234 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.234+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "68eab466-4b6b-406f-86f7-6414a3c00a32", "id": 2} v 0) v1 -- 0x7f2e6815fff0 con 0x7f2e680cf210 2023-10-16T08:15:58.239 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e657fa640 1 -- 172.21.15.186:0/4042318050 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68eab466-4b6b-406f-86f7-6414a3c00a32", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e5c02a070 con 0x7f2e680cf210 2023-10-16T08:15:58.239 INFO:teuthology.orchestra.run.smithi186.stdout:2 2023-10-16T08:15:58.240 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 msgr2=0x7f2e6815af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.240 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 0x7f2e6815af40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00e930 tx=0x7f2e5c00edf0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.241 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 shutdown_connections 2023-10-16T08:15:58.241 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e6815cdc0 0x7f2e681f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.241 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e680d3320 0x7f2e681f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.241 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 --2- 172.21.15.186:0/4042318050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e680cf210 0x7f2e6815af40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 >> 172.21.15.186:0/4042318050 conn(0x7f2e680d2800 msgr2=0x7f2e680af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 shutdown_connections 2023-10-16T08:15:58.242 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.238+0000 7f2e6dba7640 1 -- 172.21.15.186:0/4042318050 wait complete. 2023-10-16T08:15:58.258 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 7376c799-427b-4894-9d89-d4ffcff9510e 3 2023-10-16T08:15:58.394 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 Processor -- start 2023-10-16T08:15:58.394 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 -- start start 2023-10-16T08:15:58.395 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c40b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5c40d1fa0 0x7fd5c40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c415d010 con 0x7fd5c40cd840 2023-10-16T08:15:58.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c40b74d0 con 0x7fd5c40d1fa0 2023-10-16T08:15:58.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5cbc91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40b77b0 con 0x7fd5c415cc30 2023-10-16T08:15:58.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c922c640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fd5c40cd840 0x7fd5c40cdc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:47174/0 (socket says 172.21.15.186:47174) 2023-10-16T08:15:58.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c922c640 1 -- 172.21.15.186:0/2490565147 learned_addr learned my addr 172.21.15.186:0/2490565147 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-10-16T08:15:58.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c9a2d640 1 --2- 172.21.15.186:0/2490565147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c40b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5ca22e640 1 --2- 172.21.15.186:0/2490565147 >> v2:172.21.15.32:3300/0 conn(0x7fd5c40d1fa0 0x7fd5c40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c9a2d640 1 -- 172.21.15.186:0/2490565147 >> v2:172.21.15.32:3300/0 conn(0x7fd5c40d1fa0 msgr2=0x7fd5c40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:58.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c9a2d640 1 --2- 172.21.15.186:0/2490565147 >> v2:172.21.15.32:3300/0 conn(0x7fd5c40d1fa0 0x7fd5c40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c9a2d640 1 -- 172.21.15.186:0/2490565147 >> v1:172.21.15.186:6789/0 conn(0x7fd5c40cd840 legacy=0x7fd5c40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.398 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.394+0000 7fd5c9a2d640 1 -- 172.21.15.186:0/2490565147 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c40b7a90 con 0x7fd5c415cc30 2023-10-16T08:15:58.399 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5c9a2d640 1 --2- 172.21.15.186:0/2490565147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c40b8780 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00097c0 tx=0x7fd5c001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13da99b819d9ed34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.399 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5c8a2b640 1 -- 172.21.15.186:0/2490565147 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00ab670 con 0x7fd5c415cc30 2023-10-16T08:15:58.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5c8a2b640 1 -- 172.21.15.186:0/2490565147 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c00ab7d0 con 0x7fd5c415cc30 2023-10-16T08:15:58.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5c8a2b640 1 -- 172.21.15.186:0/2490565147 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00abb60 con 0x7fd5c415cc30 2023-10-16T08:15:58.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 -- 172.21.15.186:0/2490565147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 msgr2=0x7fd5c40b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/2490565147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c40b8780 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00097c0 tx=0x7fd5c001c450 comp rx=0 tx=0).stop 2023-10-16T08:15:58.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 -- 172.21.15.186:0/2490565147 shutdown_connections 2023-10-16T08:15:58.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/2490565147 >> v2:172.21.15.32:3300/0 conn(0x7fd5c40d1fa0 0x7fd5c40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/2490565147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c40b8780 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 -- 172.21.15.186:0/2490565147 >> 172.21.15.186:0/2490565147 conn(0x7fd5c406b750 msgr2=0x7fd5c406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 -- 172.21.15.186:0/2490565147 shutdown_connections 2023-10-16T08:15:58.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.398+0000 7fd5cbc91640 1 -- 172.21.15.186:0/2490565147 wait complete. 2023-10-16T08:15:58.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 Processor -- start 2023-10-16T08:15:58.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 -- start start 2023-10-16T08:15:58.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5c40cd840 0x7fd5c4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c40d1fa0 0x7fd5c4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4154250 con 0x7fd5c40d1fa0 2023-10-16T08:15:58.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c41544f0 con 0x7fd5c40cd840 2023-10-16T08:15:58.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c41547d0 con 0x7fd5c415cc30 2023-10-16T08:15:58.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5c40cd840 0x7fd5c4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ca22e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ca22e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34620/0 (socket says 172.21.15.186:34620) 2023-10-16T08:15:58.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ca22e640 1 -- 172.21.15.186:0/1649655679 learned_addr learned my addr 172.21.15.186:0/1649655679 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:58.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c922c640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c40d1fa0 0x7fd5c4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 -- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 msgr2=0x7fd5c415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.407 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 -- 172.21.15.186:0/1649655679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c40d1fa0 msgr2=0x7fd5c4153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.407 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c40d1fa0 0x7fd5c4153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.407 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 -- 172.21.15.186:0/1649655679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0009420 con 0x7fd5c40cd840 2023-10-16T08:15:58.407 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ca22e640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:58.407 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5c9a2d640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5c40cd840 0x7fd5c4158e80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00ac000 tx=0x7fd5c00191f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.408 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00bf650 con 0x7fd5c40cd840 2023-10-16T08:15:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c00bf7d0 con 0x7fd5c40cd840 2023-10-16T08:15:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c4154ab0 con 0x7fd5c40cd840 2023-10-16T08:15:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00bfb80 con 0x7fd5c40cd840 2023-10-16T08:15:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c4155070 con 0x7fd5c40cd840 2023-10-16T08:15:58.409 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd5c00bfce0 con 0x7fd5c40cd840 2023-10-16T08:15:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.402+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd5c00b43f0 con 0x7fd5c40cd840 2023-10-16T08:15:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.406+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c4154910 con 0x7fd5c40cd840 2023-10-16T08:15:58.410 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.406+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd5c4154910 con 0x7fd5c40cd840 2023-10-16T08:15:58.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.474+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7376c799-427b-4894-9d89-d4ffcff9510e", "id": 3} v 0) v1 -- 0x7fd5c4067230 con 0x7fd5c40cd840 2023-10-16T08:15:58.478 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5ba7fc640 1 -- 172.21.15.186:0/1649655679 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7376c799-427b-4894-9d89-d4ffcff9510e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd5c00a7070 con 0x7fd5c40cd840 2023-10-16T08:15:58.479 INFO:teuthology.orchestra.run.smithi186.stdout:3 2023-10-16T08:15:58.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5c40cd840 msgr2=0x7fd5c4158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5c40cd840 0x7fd5c4158e80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00ac000 tx=0x7fd5c00191f0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 shutdown_connections 2023-10-16T08:15:58.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5c415cc30 0x7fd5c415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c40d1fa0 0x7fd5c4153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 --2- 172.21.15.186:0/1649655679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5c40cd840 0x7fd5c4158e80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 >> 172.21.15.186:0/1649655679 conn(0x7fd5c406b750 msgr2=0x7fd5c406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 shutdown_connections 2023-10-16T08:15:58.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.478+0000 7fd5cbc91640 1 -- 172.21.15.186:0/1649655679 wait complete. 2023-10-16T08:15:58.498 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new d004f570-0acb-458d-a7e3-d99e637baa3d 4 2023-10-16T08:15:58.543 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:58.540+0000 7ff82880c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T08:15:58.633 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:58.634+0000 7f4fc5d63180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T08:15:58.635 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 Processor -- start 2023-10-16T08:15:58.635 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 -- start start 2023-10-16T08:15:58.637 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa77815cba0 0x7fa7780d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.638 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7780b84e0 0x7fa7780cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.638 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7780d1670 con 0x7fa7780d0040 2023-10-16T08:15:58.638 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7780d1460 con 0x7fa7780b84e0 2023-10-16T08:15:58.638 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77eb35640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780d04b0 con 0x7fa77815cba0 2023-10-16T08:15:58.638 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77c8d1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa77815cba0 0x7fa7780d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77c8d1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa77815cba0 0x7fa7780d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34626/0 (socket says 172.21.15.186:34626) 2023-10-16T08:15:58.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77d0d2640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fa7780d0040 0x7fa7780d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:47180/0 (socket says 172.21.15.186:47180) 2023-10-16T08:15:58.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa77c8d1640 1 -- 172.21.15.186:0/1823586654 learned_addr learned my addr 172.21.15.186:0/1823586654 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:58.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa777fff640 1 --2- 172.21.15.186:0/1823586654 >> v2:172.21.15.32:3300/0 conn(0x7fa7780b84e0 0x7fa7780cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa7777fe640 1 -- 172.21.15.186:0/1823586654 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926727269 0 0) 0x7fa7780d1670 con 0x7fa7780d0040 2023-10-16T08:15:58.639 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.634+0000 7fa7777fe640 1 -- 172.21.15.186:0/1823586654 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa768003530 con 0x7fa7780d0040 2023-10-16T08:15:58.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa777fff640 1 -- 172.21.15.186:0/1823586654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa77815cba0 msgr2=0x7fa7780d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:58.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa777fff640 1 --2- 172.21.15.186:0/1823586654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa77815cba0 0x7fa7780d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa777fff640 1 -- 172.21.15.186:0/1823586654 >> v1:172.21.15.186:6789/0 conn(0x7fa7780d0040 legacy=0x7fa7780d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.640 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa777fff640 1 -- 172.21.15.186:0/1823586654 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7780d1950 con 0x7fa7780b84e0 2023-10-16T08:15:58.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa777fff640 1 --2- 172.21.15.186:0/1823586654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7780b84e0 0x7fa7780cfb00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa7640097c0 tx=0x7fa76401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2069c59d7ef0a69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa7777fe640 1 -- 172.21.15.186:0/1823586654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640ac600 con 0x7fa7780b84e0 2023-10-16T08:15:58.641 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa7777fe640 1 -- 172.21.15.186:0/1823586654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7640a8050 con 0x7fa7780b84e0 2023-10-16T08:15:58.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.638+0000 7fa7777fe640 1 -- 172.21.15.186:0/1823586654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640ac980 con 0x7fa7780b84e0 2023-10-16T08:15:58.642 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- 172.21.15.186:0/1823586654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7780b84e0 msgr2=0x7fa7780cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 --2- 172.21.15.186:0/1823586654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7780b84e0 0x7fa7780cfb00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa7640097c0 tx=0x7fa76401c480 comp rx=0 tx=0).stop 2023-10-16T08:15:58.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- 172.21.15.186:0/1823586654 shutdown_connections 2023-10-16T08:15:58.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 --2- 172.21.15.186:0/1823586654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7780b84e0 0x7fa7780cfb00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 --2- 172.21.15.186:0/1823586654 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa77815cba0 0x7fa7780d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.643 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- 172.21.15.186:0/1823586654 >> 172.21.15.186:0/1823586654 conn(0x7fa7780719c0 msgr2=0x7fa778071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.644 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- 172.21.15.186:0/1823586654 shutdown_connections 2023-10-16T08:15:58.644 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- 172.21.15.186:0/1823586654 wait complete. 2023-10-16T08:15:58.646 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 Processor -- start 2023-10-16T08:15:58.646 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- start start 2023-10-16T08:15:58.647 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 0x7fa778156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.647 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 0x7fa77815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.647 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77815cba0 0x7fa77815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.647 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa778155f50 con 0x7fa77815cba0 2023-10-16T08:15:58.647 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781552d0 con 0x7fa7780b84e0 2023-10-16T08:15:58.648 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77eb35640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781555b0 con 0x7fa7780d0040 2023-10-16T08:15:58.648 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77c8d1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 0x7fa778156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.648 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa777fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 0x7fa77815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.648 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa777fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 0x7fa77815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34636/0 (socket says 172.21.15.186:34636) 2023-10-16T08:15:58.648 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa777fff640 1 -- 172.21.15.186:0/481560947 learned_addr learned my addr 172.21.15.186:0/481560947 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:58.649 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77c8d1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 0x7fa778156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:34512/0 (socket says 172.21.15.186:34512) 2023-10-16T08:15:58.649 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.642+0000 7fa77d0d2640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77815cba0 0x7fa77815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.649 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77c8d1640 1 -- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 msgr2=0x7fa77815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.649 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77c8d1640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 0x7fa77815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.649 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77c8d1640 1 -- 172.21.15.186:0/481560947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77815cba0 msgr2=0x7fa77815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.650 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77c8d1640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77815cba0 0x7fa77815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.650 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77c8d1640 1 -- 172.21.15.186:0/481560947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009420 con 0x7fa7780b84e0 2023-10-16T08:15:58.650 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa777fff640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 0x7fa77815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:58.650 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77c8d1640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 0x7fa778156930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e870 tx=0x7fa76c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.650 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00ee60 con 0x7fa7780b84e0 2023-10-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c00c940 con 0x7fa7780b84e0 2023-10-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa778155890 con 0x7fa7780b84e0 2023-10-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00ccd0 con 0x7fa7780b84e0 2023-10-16T08:15:58.651 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7780cb2b0 con 0x7fa7780b84e0 2023-10-16T08:15:58.652 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa76c005ca0 con 0x7fa7780b84e0 2023-10-16T08:15:58.652 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa76c00ce30 con 0x7fa7780b84e0 2023-10-16T08:15:58.652 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.646+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7780b6cc0 con 0x7fa7780b84e0 2023-10-16T08:15:58.652 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.650+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa76c021060 con 0x7fa7780b84e0 2023-10-16T08:15:58.699 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:58.698+0000 7f4fc5d63180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T08:15:58.715 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.714+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d004f570-0acb-458d-a7e3-d99e637baa3d", "id": 4} v 0) v1 -- 0x7fa77804e170 con 0x7fa7780b84e0 2023-10-16T08:15:58.720 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.718+0000 7fa7757fa640 1 -- 172.21.15.186:0/481560947 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d004f570-0acb-458d-a7e3-d99e637baa3d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa76c029020 con 0x7fa7780b84e0 2023-10-16T08:15:58.721 INFO:teuthology.orchestra.run.smithi186.stdout:4 2023-10-16T08:15:58.722 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 msgr2=0x7fa778156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.722 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 0x7fa778156930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e870 tx=0x7fa76c00ed30 comp rx=0 tx=0).stop 2023-10-16T08:15:58.722 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 shutdown_connections 2023-10-16T08:15:58.723 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77815cba0 0x7fa77815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.723 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7780d0040 0x7fa77815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.723 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 --2- 172.21.15.186:0/481560947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7780b84e0 0x7fa778156930 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.723 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 >> 172.21.15.186:0/481560947 conn(0x7fa7780719c0 msgr2=0x7fa778071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.723 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 shutdown_connections 2023-10-16T08:15:58.723 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.722+0000 7fa77eb35640 1 -- 172.21.15.186:0/481560947 wait complete. 2023-10-16T08:15:58.740 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 42ff7c69-5065-4349-98ef-e35cd5a7a0fb 5 2023-10-16T08:15:58.758 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:58.756+0000 7ff82880c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T08:15:58.763 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:58.762+0000 7f4fc5d63180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T08:15:58.854 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:58.854+0000 7f4fc5d63180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T08:15:58.877 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.874+0000 7f4e4b79d640 1 Processor -- start 2023-10-16T08:15:58.877 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.874+0000 7f4e4b79d640 1 -- start start 2023-10-16T08:15:58.878 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e4b79d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e440d0210 0x7f4e440d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.879 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e4b79d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e440d1670 con 0x7f4e4415cba0 2023-10-16T08:15:58.879 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e4b79d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e440d1950 con 0x7f4e440b84e0 2023-10-16T08:15:58.879 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e4b79d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e440d1460 con 0x7f4e440d0210 2023-10-16T08:15:58.879 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e49d3a640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f4e440b84e0 0x7f4e440b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:47184/0 (socket says 172.21.15.186:47184) 2023-10-16T08:15:58.880 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e49d3a640 1 -- 172.21.15.186:0/1393131199 learned_addr learned my addr 172.21.15.186:0/1393131199 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-10-16T08:15:58.880 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e48d38640 1 --2- 172.21.15.186:0/1393131199 >> v2:172.21.15.32:3300/0 conn(0x7f4e440d0210 0x7f4e440d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.881 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e3bfff640 1 -- 172.21.15.186:0/1393131199 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067805088 0 0) 0x7f4e440d1950 con 0x7f4e440b84e0 2023-10-16T08:15:58.881 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e3bfff640 1 -- 172.21.15.186:0/1393131199 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e30003530 con 0x7f4e440b84e0 2023-10-16T08:15:58.881 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e48d38640 1 -- 172.21.15.186:0/1393131199 >> v1:172.21.15.186:6789/0 conn(0x7f4e440b84e0 legacy=0x7f4e440b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.881 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e3bfff640 1 -- 172.21.15.186:0/1393131199 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1080678570 0 0) 0x7f4e30003530 con 0x7f4e440b84e0 2023-10-16T08:15:58.881 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e48d38640 1 -- 172.21.15.186:0/1393131199 >> v1:172.21.15.32:6789/0 conn(0x7f4e4415cba0 legacy=0x7f4e440cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.882 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.878+0000 7f4e48d38640 1 -- 172.21.15.186:0/1393131199 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440d1c30 con 0x7f4e440d0210 2023-10-16T08:15:58.882 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e48d38640 1 --2- 172.21.15.186:0/1393131199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440d0210 0x7f4e440d0f20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e340097c0 tx=0x7f4e3401c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b4f4fcb7af6e15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.883 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e3bfff640 1 -- 172.21.15.186:0/1393131199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c001a00 con 0x7f4e440d0210 2023-10-16T08:15:58.883 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e3bfff640 1 -- 172.21.15.186:0/1393131199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e340a7050 con 0x7f4e440d0210 2023-10-16T08:15:58.883 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e3bfff640 1 -- 172.21.15.186:0/1393131199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e340ab620 con 0x7f4e440d0210 2023-10-16T08:15:58.884 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1393131199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440d0210 msgr2=0x7f4e440d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.884 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 --2- 172.21.15.186:0/1393131199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440d0210 0x7f4e440d0f20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e340097c0 tx=0x7f4e3401c1c0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.884 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1393131199 shutdown_connections 2023-10-16T08:15:58.885 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 --2- 172.21.15.186:0/1393131199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440d0210 0x7f4e440d0f20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.885 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1393131199 >> 172.21.15.186:0/1393131199 conn(0x7f4e440719c0 msgr2=0x7f4e44071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.885 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1393131199 shutdown_connections 2023-10-16T08:15:58.885 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1393131199 wait complete. 2023-10-16T08:15:58.888 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.882+0000 7f4e4b79d640 1 Processor -- start 2023-10-16T08:15:58.888 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- start start 2023-10-16T08:15:58.888 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440b84e0 0x7f4e440caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.889 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e440d0210 0x7f4e440ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.889 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 0x7f4e44154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:58.889 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440c9f90 con 0x7f4e440b84e0 2023-10-16T08:15:58.889 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441554a0 con 0x7f4e4415cba0 2023-10-16T08:15:58.890 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440cb4a0 con 0x7f4e440d0210 2023-10-16T08:15:58.890 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e49d3a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 0x7f4e44154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.890 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e49d3a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 0x7f4e44154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:34524/0 (socket says 172.21.15.186:34524) 2023-10-16T08:15:58.890 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e440d0210 0x7f4e440ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.890 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e49d3a640 1 -- 172.21.15.186:0/1984907819 learned_addr learned my addr 172.21.15.186:0/1984907819 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:58.891 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e49539640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440b84e0 0x7f4e440caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:58.891 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 -- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 msgr2=0x7f4e44154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.891 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 0x7f4e44154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.891 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 -- 172.21.15.186:0/1984907819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440b84e0 msgr2=0x7f4e440caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.891 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440b84e0 0x7f4e440caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.892 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 -- 172.21.15.186:0/1984907819 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e34009420 con 0x7f4e440d0210 2023-10-16T08:15:58.892 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e49d3a640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 0x7f4e44154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:58.892 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e48d38640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e440d0210 0x7f4e440ca420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e340ab000 tx=0x7f4e3401aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:58.892 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e340b7450 con 0x7f4e440d0210 2023-10-16T08:15:58.892 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e340b75b0 con 0x7f4e440d0210 2023-10-16T08:15:58.893 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e340b7940 con 0x7f4e440d0210 2023-10-16T08:15:58.893 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440cb770 con 0x7f4e440d0210 2023-10-16T08:15:58.893 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e440cbd30 con 0x7f4e440d0210 2023-10-16T08:15:58.893 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e340c0400 con 0x7f4e440d0210 2023-10-16T08:15:58.893 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f4e340b7da0 con 0x7f4e440d0210 2023-10-16T08:15:58.894 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.886+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e440cbf20 con 0x7f4e440d0210 2023-10-16T08:15:58.894 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.890+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4e340bb020 con 0x7f4e440d0210 2023-10-16T08:15:58.917 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:58.914+0000 7f4fc5d63180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T08:15:58.958 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.958+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd new", "uuid": "42ff7c69-5065-4349-98ef-e35cd5a7a0fb", "id": 5} v 0) v1 -- 0x7f4e4404e170 con 0x7f4e440d0210 2023-10-16T08:15:58.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e39ffb640 1 -- 172.21.15.186:0/1984907819 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42ff7c69-5065-4349-98ef-e35cd5a7a0fb", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e340b0070 con 0x7f4e440d0210 2023-10-16T08:15:58.962 INFO:teuthology.orchestra.run.smithi186.stdout:5 2023-10-16T08:15:58.963 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e440d0210 msgr2=0x7f4e440ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:58.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e440d0210 0x7f4e440ca420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e340ab000 tx=0x7f4e3401aa00 comp rx=0 tx=0).stop 2023-10-16T08:15:58.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 shutdown_connections 2023-10-16T08:15:58.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e4415cba0 0x7f4e44154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e440d0210 0x7f4e440ca420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 --2- 172.21.15.186:0/1984907819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e440b84e0 0x7f4e440caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:58.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 >> 172.21.15.186:0/1984907819 conn(0x7f4e440719c0 msgr2=0x7f4e4415a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:58.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 shutdown_connections 2023-10-16T08:15:58.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:58.962+0000 7f4e4b79d640 1 -- 172.21.15.186:0/1984907819 wait complete. 2023-10-16T08:15:58.982 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 2e1f98cc-738b-4c3d-b766-7a38b271ddf3 6 2023-10-16T08:15:59.010 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:59.009+0000 7f4fc5d63180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T08:15:59.071 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:59.069+0000 7f4fc5d63180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T08:15:59.146 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 Processor -- start 2023-10-16T08:15:59.147 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 -- start start 2023-10-16T08:15:59.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f280d1fa0 0x7f4f280b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4f280cdf50 0x7f4f280b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2815d010 con 0x7f4f2815cc30 2023-10-16T08:15:59.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f280b74d0 con 0x7f4f280cdf50 2023-10-16T08:15:59.150 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2f53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280b77b0 con 0x7f4f280d1fa0 2023-10-16T08:15:59.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2d2da640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f4f2815cc30 0x7f4f280cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.186:47188/0 (socket says 172.21.15.186:47188) 2023-10-16T08:15:59.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2d2da640 1 -- 172.21.15.186:0/39640694 learned_addr learned my addr 172.21.15.186:0/39640694 (peer_addr_for_me v1:172.21.15.186:0/0) 2023-10-16T08:15:59.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2cad9640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f280d1fa0 0x7f4f280b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.145+0000 7f4f2dadb640 1 --2- 172.21.15.186:0/39640694 >> v2:172.21.15.186:3300/0 conn(0x7f4f280cdf50 0x7f4f280b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f2dadb640 1 -- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f280d1fa0 msgr2=0x7f4f280b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:59.151 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f2dadb640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f280d1fa0 0x7f4f280b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f2dadb640 1 -- 172.21.15.186:0/39640694 >> v1:172.21.15.186:6789/0 conn(0x7f4f2815cc30 legacy=0x7f4f280cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f2dadb640 1 -- 172.21.15.186:0/39640694 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f280b7a90 con 0x7f4f280cdf50 2023-10-16T08:15:59.152 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f2cad9640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f280d1fa0 0x7f4f280b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:59.153 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f2dadb640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280b5be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009d90 tx=0x7f4f2001f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c46bd0eff37251 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:59.153 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f1ffff640 1 -- 172.21.15.186:0/39640694 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200af600 con 0x7f4f280cdf50 2023-10-16T08:15:59.153 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f1ffff640 1 -- 172.21.15.186:0/39640694 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f200ab050 con 0x7f4f280cdf50 2023-10-16T08:15:59.153 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.149+0000 7f4f1ffff640 1 -- 172.21.15.186:0/39640694 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200af980 con 0x7f4f280cdf50 2023-10-16T08:15:59.154 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 msgr2=0x7f4f280b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280b5be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009d90 tx=0x7f4f2001f130 comp rx=0 tx=0).stop 2023-10-16T08:15:59.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- 172.21.15.186:0/39640694 shutdown_connections 2023-10-16T08:15:59.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f280d1fa0 0x7f4f280b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/39640694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280b5be0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- 172.21.15.186:0/39640694 >> 172.21.15.186:0/39640694 conn(0x7f4f2806b750 msgr2=0x7f4f2806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:59.155 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- 172.21.15.186:0/39640694 shutdown_connections 2023-10-16T08:15:59.156 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- 172.21.15.186:0/39640694 wait complete. 2023-10-16T08:15:59.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 Processor -- start 2023-10-16T08:15:59.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- start start 2023-10-16T08:15:59.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f280d1fa0 0x7f4f281f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f2815cc30 0x7f4f280d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280ca5e0 con 0x7f4f280d1fa0 2023-10-16T08:15:59.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f281f2d00 con 0x7f4f280cdf50 2023-10-16T08:15:59.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2f53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280cbde0 con 0x7f4f2815cc30 2023-10-16T08:15:59.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:34538/0 (socket says 172.21.15.186:34538) 2023-10-16T08:15:59.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 -- 172.21.15.186:0/2936995207 learned_addr learned my addr 172.21.15.186:0/2936995207 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:59.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2dadb640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f2815cc30 0x7f4f280d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2cad9640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f280d1fa0 0x7f4f281f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 -- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f2815cc30 msgr2=0x7f4f280d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f2815cc30 0x7f4f280d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 -- 172.21.15.186:0/2936995207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f280d1fa0 msgr2=0x7f4f281f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.162 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f280d1fa0 0x7f4f281f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.153+0000 7f4f2d2da640 1 -- 172.21.15.186:0/2936995207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009420 con 0x7f4f280cdf50 2023-10-16T08:15:59.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f2cad9640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f280d1fa0 0x7f4f281f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:15:59.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f2d2da640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280cb720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c400 tx=0x7f4f2400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:59.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24005cf0 con 0x7f4f280cdf50 2023-10-16T08:15:59.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24005e50 con 0x7f4f280cdf50 2023-10-16T08:15:59.164 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240072d0 con 0x7f4f280cdf50 2023-10-16T08:15:59.164 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f20009a70 con 0x7f4f280cdf50 2023-10-16T08:15:59.164 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4f24012400 con 0x7f4f280cdf50 2023-10-16T08:15:59.164 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f280c9f20 con 0x7f4f280cdf50 2023-10-16T08:15:59.164 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4f2401b450 con 0x7f4f280cdf50 2023-10-16T08:15:59.165 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.157+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f28067230 con 0x7f4f280cdf50 2023-10-16T08:15:59.165 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.161+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4f24017070 con 0x7f4f280cdf50 2023-10-16T08:15:59.228 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.225+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2e1f98cc-738b-4c3d-b766-7a38b271ddf3", "id": 6} v 0) v1 -- 0x7f4f280d0960 con 0x7f4f280cdf50 2023-10-16T08:15:59.232 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.229+0000 7f4f1dffb640 1 -- 172.21.15.186:0/2936995207 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e1f98cc-738b-4c3d-b766-7a38b271ddf3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4f240128e0 con 0x7f4f280cdf50 2023-10-16T08:15:59.233 INFO:teuthology.orchestra.run.smithi186.stdout:6 2023-10-16T08:15:59.234 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 msgr2=0x7f4f280cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.234 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280cb720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c400 tx=0x7f4f2400c8c0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.234 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 shutdown_connections 2023-10-16T08:15:59.234 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4f2815cc30 0x7f4f280d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.235 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f280d1fa0 0x7f4f281f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.235 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 --2- 172.21.15.186:0/2936995207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f280cdf50 0x7f4f280cb720 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.235 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 >> 172.21.15.186:0/2936995207 conn(0x7f4f2806b750 msgr2=0x7f4f280b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:59.235 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 shutdown_connections 2023-10-16T08:15:59.236 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4f2f53e640 1 -- 172.21.15.186:0/2936995207 wait complete. 2023-10-16T08:15:59.236 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:59.233+0000 7f4fc5d63180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T08:15:59.252 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 6ef75fcd-dce3-4d95-b560-d54fc2be1226 7 2023-10-16T08:15:59.306 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:15:59.305+0000 7f4fc5d63180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T08:15:59.388 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.385+0000 7fdd88b3e640 1 Processor -- start 2023-10-16T08:15:59.389 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.385+0000 7fdd88b3e640 1 -- start start 2023-10-16T08:15:59.391 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd88b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 0x7fdd840b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.391 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd88b3e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd840d1fa0 0x7fdd840b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.391 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd88b3e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8415d010 con 0x7fdd840cd840 2023-10-16T08:15:59.391 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd88b3e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd840b74d0 con 0x7fdd840d1fa0 2023-10-16T08:15:59.392 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd88b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840b77b0 con 0x7fdd8415cc30 2023-10-16T08:15:59.392 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 0x7fdd840b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.392 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 0x7fdd840b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34670/0 (socket says 172.21.15.186:34670) 2023-10-16T08:15:59.392 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 -- 172.21.15.186:0/961186566 learned_addr learned my addr 172.21.15.186:0/961186566 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:59.393 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd82d9d640 1 --2- 172.21.15.186:0/961186566 >> v2:172.21.15.32:3300/0 conn(0x7fdd840d1fa0 0x7fdd840b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.393 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 -- 172.21.15.186:0/961186566 >> v2:172.21.15.32:3300/0 conn(0x7fdd840d1fa0 msgr2=0x7fdd840b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:59.393 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 --2- 172.21.15.186:0/961186566 >> v2:172.21.15.32:3300/0 conn(0x7fdd840d1fa0 0x7fdd840b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.393 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 -- 172.21.15.186:0/961186566 >> v1:172.21.15.32:6789/0 conn(0x7fdd840cd840 legacy=0x7fdd840cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.393 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.389+0000 7fdd8259c640 1 -- 172.21.15.186:0/961186566 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840b7a90 con 0x7fdd8415cc30 2023-10-16T08:15:59.395 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd8259c640 1 --2- 172.21.15.186:0/961186566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 0x7fdd840b8780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009870 tx=0x7fdd7401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60d3da9a661baa14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:59.395 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd8159a640 1 -- 172.21.15.186:0/961186566 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740a95f0 con 0x7fdd8415cc30 2023-10-16T08:15:59.395 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd8159a640 1 -- 172.21.15.186:0/961186566 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd740a9750 con 0x7fdd8415cc30 2023-10-16T08:15:59.395 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd8159a640 1 -- 172.21.15.186:0/961186566 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740a9ae0 con 0x7fdd8415cc30 2023-10-16T08:15:59.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 -- 172.21.15.186:0/961186566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 msgr2=0x7fdd840b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/961186566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 0x7fdd840b8780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009870 tx=0x7fdd7401c510 comp rx=0 tx=0).stop 2023-10-16T08:15:59.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 -- 172.21.15.186:0/961186566 shutdown_connections 2023-10-16T08:15:59.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/961186566 >> v2:172.21.15.32:3300/0 conn(0x7fdd840d1fa0 0x7fdd840b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.396 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/961186566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd8415cc30 0x7fdd840b8780 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 -- 172.21.15.186:0/961186566 >> 172.21.15.186:0/961186566 conn(0x7fdd8406b750 msgr2=0x7fdd8406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:59.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 -- 172.21.15.186:0/961186566 shutdown_connections 2023-10-16T08:15:59.397 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 -- 172.21.15.186:0/961186566 wait complete. 2023-10-16T08:15:59.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 Processor -- start 2023-10-16T08:15:59.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.393+0000 7fdd88b3e640 1 -- start start 2023-10-16T08:15:59.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 0x7fdd84158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 0x7fdd84153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.400 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8415cc30 0x7fdd8415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84154250 con 0x7fdd8415cc30 2023-10-16T08:15:59.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd841544f0 con 0x7fdd840d1fa0 2023-10-16T08:15:59.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd841547d0 con 0x7fdd840cd840 2023-10-16T08:15:59.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd8259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 0x7fdd84158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd81d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 0x7fdd84153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd8259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 0x7fdd84158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.186:34678/0 (socket says 172.21.15.186:34678) 2023-10-16T08:15:59.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd81d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 0x7fdd84153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:34550/0 (socket says 172.21.15.186:34550) 2023-10-16T08:15:59.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd8259c640 1 -- 172.21.15.186:0/1342502855 learned_addr learned my addr 172.21.15.186:0/1342502855 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T08:15:59.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8415cc30 0x7fdd8415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.402 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 -- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 msgr2=0x7fdd84158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 0x7fdd84158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 -- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 msgr2=0x7fdd84153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 0x7fdd84153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 -- 172.21.15.186:0/1342502855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740094d0 con 0x7fdd8415cc30 2023-10-16T08:15:59.403 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd8259c640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 0x7fdd84158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:59.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd81d9b640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 0x7fdd84153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:15:59.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd82d9d640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8415cc30 0x7fdd8415b310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00efc0 tx=0x7fdd7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:59.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c009c00 con 0x7fdd8415cc30 2023-10-16T08:15:59.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c00f040 con 0x7fdd8415cc30 2023-10-16T08:15:59.404 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0146c0 con 0x7fdd8415cc30 2023-10-16T08:15:59.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd84154ab0 con 0x7fdd8415cc30 2023-10-16T08:15:59.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd84155070 con 0x7fdd8415cc30 2023-10-16T08:15:59.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdd7c014b20 con 0x7fdd8415cc30 2023-10-16T08:15:59.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fdd7c014dd0 con 0x7fdd8415cc30 2023-10-16T08:15:59.405 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.397+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd84155260 con 0x7fdd8415cc30 2023-10-16T08:15:59.406 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.401+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fdd7c03bb40 con 0x7fdd8415cc30 2023-10-16T08:15:59.412 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.408+0000 7ff82880c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T08:15:59.468 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.465+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ef75fcd-dce3-4d95-b560-d54fc2be1226", "id": 7} v 0) v1 -- 0x7fdd84067230 con 0x7fdd8415cc30 2023-10-16T08:15:59.471 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd72ffd640 1 -- 172.21.15.186:0/1342502855 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ef75fcd-dce3-4d95-b560-d54fc2be1226", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdd7c02a070 con 0x7fdd8415cc30 2023-10-16T08:15:59.471 INFO:teuthology.orchestra.run.smithi186.stdout:7 2023-10-16T08:15:59.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8415cc30 msgr2=0x7fdd8415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.472 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8415cc30 0x7fdd8415b310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00efc0 tx=0x7fdd7c00c490 comp rx=0 tx=0).stop 2023-10-16T08:15:59.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 shutdown_connections 2023-10-16T08:15:59.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8415cc30 0x7fdd8415b310 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd840d1fa0 0x7fdd84153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.473 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 --2- 172.21.15.186:0/1342502855 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd840cd840 0x7fdd84158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.474 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 >> 172.21.15.186:0/1342502855 conn(0x7fdd8406b750 msgr2=0x7fdd8406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:59.474 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 shutdown_connections 2023-10-16T08:15:59.474 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:15:59.469+0000 7fdd88b3e640 1 -- 172.21.15.186:0/1342502855 wait complete. 2023-10-16T08:15:59.477 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.476+0000 7ff82880c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T08:15:59.491 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-16T08:15:59.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-16T08:15:59.495 INFO:tasks.ceph.osd.0:Started 2023-10-16T08:15:59.495 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-16T08:15:59.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-16T08:15:59.499 INFO:tasks.ceph.osd.1:Started 2023-10-16T08:15:59.499 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-16T08:15:59.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-16T08:15:59.503 INFO:tasks.ceph.osd.2:Started 2023-10-16T08:15:59.503 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-16T08:15:59.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-16T08:15:59.509 INFO:tasks.ceph.osd.3:Started 2023-10-16T08:15:59.509 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-16T08:15:59.509 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-16T08:15:59.513 INFO:tasks.ceph.osd.4:Started 2023-10-16T08:15:59.513 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-16T08:15:59.514 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-16T08:15:59.517 INFO:tasks.ceph.osd.5:Started 2023-10-16T08:15:59.518 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-16T08:15:59.518 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-16T08:15:59.521 INFO:tasks.ceph.osd.6:Started 2023-10-16T08:15:59.522 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-16T08:15:59.522 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-16T08:15:59.525 INFO:tasks.ceph.osd.7:Started 2023-10-16T08:15:59.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:15:59.554 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.552+0000 7ff82880c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T08:15:59.676 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.672+0000 7ff82880c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T08:15:59.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.700+0000 7f18d7268640 1 Processor -- start 2023-10-16T08:15:59.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.700+0000 7f18d7268640 1 -- start start 2023-10-16T08:15:59.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d7268640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18d015cb90 0x7f18d00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d7268640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f18d00d15d0 0x7f18d00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d7268640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d00ce770 con 0x7f18d00d3010 2023-10-16T08:15:59.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d7268640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00d0600 con 0x7f18d015cb90 2023-10-16T08:15:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d7268640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00c6450 con 0x7f18d00d15d0 2023-10-16T08:15:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5805640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f18d00d15d0 0x7f18d00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5004640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18d015cb90 0x7f18d00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d4803640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f18d00d3010 0x7f18d00d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:38978/0 (socket says 172.21.15.32:38978) 2023-10-16T08:15:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5805640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f18d00d15d0 0x7f18d00c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:39864/0 (socket says 172.21.15.32:39864) 2023-10-16T08:15:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5805640 1 -- 172.21.15.32:0/4020149408 learned_addr learned my addr 172.21.15.32:0/4020149408 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:15:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5004640 1 -- 172.21.15.32:0/4020149408 >> v2:172.21.15.186:3300/0 conn(0x7f18d00d15d0 msgr2=0x7f18d00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:15:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5004640 1 --2- 172.21.15.32:0/4020149408 >> v2:172.21.15.186:3300/0 conn(0x7f18d00d15d0 0x7f18d00c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5004640 1 -- 172.21.15.32:0/4020149408 >> v1:172.21.15.186:6789/0 conn(0x7f18d00d3010 legacy=0x7f18d00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.704+0000 7f18d5004640 1 -- 172.21.15.32:0/4020149408 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00d08a0 con 0x7f18d015cb90 2023-10-16T08:15:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d5004640 1 --2- 172.21.15.32:0/4020149408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d015cb90 0x7f18d00cfb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18d00ccee0 tx=0x7f18cc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58bf1ffd74d5f89 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18c7fff640 1 -- 172.21.15.32:0/4020149408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0ad680 con 0x7f18d015cb90 2023-10-16T08:15:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18c7fff640 1 -- 172.21.15.32:0/4020149408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc0ad7e0 con 0x7f18d015cb90 2023-10-16T08:15:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18c7fff640 1 -- 172.21.15.32:0/4020149408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0adb70 con 0x7f18d015cb90 2023-10-16T08:15:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 -- 172.21.15.32:0/4020149408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d015cb90 msgr2=0x7f18d00cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 --2- 172.21.15.32:0/4020149408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d015cb90 0x7f18d00cfb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18d00ccee0 tx=0x7f18cc01c380 comp rx=0 tx=0).stop 2023-10-16T08:15:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 -- 172.21.15.32:0/4020149408 shutdown_connections 2023-10-16T08:15:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 --2- 172.21.15.32:0/4020149408 >> v2:172.21.15.186:3300/0 conn(0x7f18d00d15d0 0x7f18d00c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 --2- 172.21.15.32:0/4020149408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d015cb90 0x7f18d00cfb10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 -- 172.21.15.32:0/4020149408 >> 172.21.15.32:0/4020149408 conn(0x7f18d009bc50 msgr2=0x7f18d00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:59.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 -- 172.21.15.32:0/4020149408 shutdown_connections 2023-10-16T08:15:59.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 -- 172.21.15.32:0/4020149408 wait complete. 2023-10-16T08:15:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 Processor -- start 2023-10-16T08:15:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.708+0000 7f18d7268640 1 -- start start 2023-10-16T08:15:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d00d15d0 0x7f18d014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 0x7f18d00b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18d015cb90 0x7f18d01500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:15:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b7700 con 0x7f18d00d3010 2023-10-16T08:15:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b6ac0 con 0x7f18d00d15d0 2023-10-16T08:15:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b6d60 con 0x7f18d015cb90 2023-10-16T08:15:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 0x7f18d00b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 0x7f18d00b7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40894/0 (socket says 172.21.15.32:40894) 2023-10-16T08:15:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 -- 172.21.15.32:0/3631654045 learned_addr learned my addr 172.21.15.32:0/3631654045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:15:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d5004640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d00d15d0 0x7f18d014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d5805640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18d015cb90 0x7f18d01500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:15:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 -- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18d015cb90 msgr2=0x7f18d01500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18d015cb90 0x7f18d01500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 -- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d00d15d0 msgr2=0x7f18d014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d00d15d0 0x7f18d014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 -- 172.21.15.32:0/3631654045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc0093c0 con 0x7f18d00d3010 2023-10-16T08:15:59.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d4803640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 0x7f18d00b7b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18c000c460 tx=0x7f18c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:15:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0007480 con 0x7f18d00d3010 2023-10-16T08:15:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0007600 con 0x7f18d00d3010 2023-10-16T08:15:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0004a30 con 0x7f18d00d3010 2023-10-16T08:15:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d00b7040 con 0x7f18d00d3010 2023-10-16T08:15:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f18c0012440 con 0x7f18d00d3010 2023-10-16T08:15:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d014fe50 con 0x7f18d00d3010 2023-10-16T08:15:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f18c001ba30 con 0x7f18d00d3010 2023-10-16T08:15:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.712+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d00b7840 con 0x7f18d00d3010 2023-10-16T08:15:59.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.716+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f18d00b7840 con 0x7f18d00d3010 2023-10-16T08:15:59.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.740+0000 7ff82880c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T08:15:59.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.772+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18d0074010 con 0x7f18d00d3010 2023-10-16T08:15:59.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18c5ffb640 1 -- 172.21.15.32:0/3631654045 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f18c0018070 con 0x7f18d00d3010 2023-10-16T08:15:59.778 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:15:59.778 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:15:59.468950+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:15:59.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 msgr2=0x7f18d00b7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:15:59.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 0x7f18d00b7b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18c000c460 tx=0x7f18c000c920 comp rx=0 tx=0).stop 2023-10-16T08:15:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 shutdown_connections 2023-10-16T08:15:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f18d015cb90 0x7f18d01500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18d00d3010 0x7f18d00b7b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 --2- 172.21.15.32:0/3631654045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d00d15d0 0x7f18d014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:15:59.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 >> 172.21.15.32:0/3631654045 conn(0x7f18d009bc50 msgr2=0x7f18d00d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:15:59.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 shutdown_connections 2023-10-16T08:15:59.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:15:59.776+0000 7f18d7268640 1 -- 172.21.15.32:0/3631654045 wait complete. 2023-10-16T08:15:59.798 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-16T08:15:59.799 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-16T08:15:59.836 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.832+0000 7ff82880c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T08:15:59.899 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:15:59.896+0000 7ff82880c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T08:15:59.978 INFO:tasks.ceph.mgr.x.smithi186.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-16T08:15:59.979 INFO:tasks.ceph.mgr.x.smithi186.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-16T08:15:59.979 INFO:tasks.ceph.mgr.x.smithi186.stderr: from numpy import show_config as show_numpy_config 2023-10-16T08:16:00.023 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:00.021+0000 7f4fc5d63180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T08:16:00.066 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:00.064+0000 7ff82880c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T08:16:00.137 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:00.136+0000 7ff82880c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T08:16:00.158 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:00.157+0000 7f4fc5d63180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:16:00.166 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-10-16T08:16:00.164+0000 7f0914507640 -1 Falling back to public interface 2023-10-16T08:16:00.174 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-10-16T08:16:00.172+0000 7f70fdc57640 -1 Falling back to public interface 2023-10-16T08:16:00.182 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-10-16T08:16:00.180+0000 7f9ab1c4f640 -1 Falling back to public interface 2023-10-16T08:16:00.183 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-10-16T08:16:00.181+0000 7f325d28e640 -1 Falling back to public interface 2023-10-16T08:16:00.183 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-10-16T08:16:00.181+0000 7fd3169ad640 -1 Falling back to public interface 2023-10-16T08:16:00.184 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-10-16T08:16:00.181+0000 7f325c24f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-10-16T08:16:00.184 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-10-16T08:16:00.181+0000 7f9bc7c29640 -1 Falling back to public interface 2023-10-16T08:16:00.185 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-10-16T08:16:00.181+0000 7fd31596e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2023-10-16T08:16:00.195 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-10-16T08:16:00.193+0000 7f7508b53640 -1 Falling back to public interface 2023-10-16T08:16:00.202 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-10-16T08:16:00.200+0000 7f94feaac640 -1 Falling back to public interface 2023-10-16T08:16:00.342 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:00.341+0000 7f4fc5d63180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T08:16:00.414 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:00.413+0000 7f4fc5d63180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T08:16:00.552 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:00.549+0000 7f4fc5d63180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:16:00.604 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:00.736 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 Processor -- start 2023-10-16T08:16:00.736 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 -- start start 2023-10-16T08:16:00.737 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc9815ca60 0x7fcc9806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:00.737 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9806b980 0x7fcc980ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:00.738 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc980cfd50 con 0x7fcc9815ce40 2023-10-16T08:16:00.738 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc980d0030 con 0x7fcc9815ca60 2023-10-16T08:16:00.738 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.732+0000 7fcc9d886640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980cd520 con 0x7fcc9806b980 2023-10-16T08:16:00.738 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc9815ca60 0x7fcc9806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:00.738 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc9815ca60 0x7fcc9806b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40898/0 (socket says 172.21.15.32:40898) 2023-10-16T08:16:00.739 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc977fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc9815ce40 0x7fcc980d0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38746/0 (socket says 172.21.15.32:38746) 2023-10-16T08:16:00.739 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 -- 172.21.15.32:0/264943285 learned_addr learned my addr 172.21.15.32:0/264943285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:00.739 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc967fc640 1 --2- 172.21.15.32:0/264943285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9806b980 0x7fcc980ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:00.740 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 -- 172.21.15.32:0/264943285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9806b980 msgr2=0x7fcc980ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:00.740 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 --2- 172.21.15.32:0/264943285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9806b980 0x7fcc980ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.740 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 -- 172.21.15.32:0/264943285 >> v1:172.21.15.32:6789/0 conn(0x7fcc9815ce40 legacy=0x7fcc980d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:00.740 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 -- 172.21.15.32:0/264943285 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980cd800 con 0x7fcc9815ca60 2023-10-16T08:16:00.741 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc95ffb640 1 -- 172.21.15.32:0/264943285 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045374212 0 0) 0x7fcc980cfd50 con 0x7fcc9815ce40 2023-10-16T08:16:00.742 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc96ffd640 1 --2- 172.21.15.32:0/264943285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9815ca60 0x7fcc9806b440 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009860 tx=0x7fcc8801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bc02d187cccdb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:00.742 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc95ffb640 1 -- 172.21.15.32:0/264943285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc880aa670 con 0x7fcc9815ca60 2023-10-16T08:16:00.742 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc95ffb640 1 -- 172.21.15.32:0/264943285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc880aa7d0 con 0x7fcc9815ca60 2023-10-16T08:16:00.743 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.736+0000 7fcc95ffb640 1 -- 172.21.15.32:0/264943285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc880aab60 con 0x7fcc9815ca60 2023-10-16T08:16:00.743 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- 172.21.15.32:0/264943285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9815ca60 msgr2=0x7fcc9806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:00.743 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 --2- 172.21.15.32:0/264943285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9815ca60 0x7fcc9806b440 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009860 tx=0x7fcc8801c4f0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.743 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- 172.21.15.32:0/264943285 shutdown_connections 2023-10-16T08:16:00.743 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 --2- 172.21.15.32:0/264943285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9806b980 0x7fcc980ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.744 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 --2- 172.21.15.32:0/264943285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9815ca60 0x7fcc9806b440 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.744 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- 172.21.15.32:0/264943285 >> 172.21.15.32:0/264943285 conn(0x7fcc9808c350 msgr2=0x7fcc9809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:00.744 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- 172.21.15.32:0/264943285 shutdown_connections 2023-10-16T08:16:00.744 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- 172.21.15.32:0/264943285 wait complete. 2023-10-16T08:16:00.746 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 Processor -- start 2023-10-16T08:16:00.747 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- start start 2023-10-16T08:16:00.747 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 0x7fcc98153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:00.747 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9815ca60 0x7fcc981535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:00.747 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc9815ce40 0x7fcc981f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:00.747 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9814ac40 con 0x7fcc9806b980 2023-10-16T08:16:00.748 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980c64b0 con 0x7fcc9815ce40 2023-10-16T08:16:00.748 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc9d886640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980c6790 con 0x7fcc9815ca60 2023-10-16T08:16:00.748 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc96ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 0x7fcc98153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:00.748 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc96ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 0x7fcc98153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40910/0 (socket says 172.21.15.32:40910) 2023-10-16T08:16:00.748 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc96ffd640 1 -- 172.21.15.32:0/3353539874 learned_addr learned my addr 172.21.15.32:0/3353539874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:00.749 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc977fe640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc9815ce40 0x7fcc981f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:00.749 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.740+0000 7fcc967fc640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9815ca60 0x7fcc981535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:00.749 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc96ffd640 1 -- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9815ca60 msgr2=0x7fcc981535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:00.749 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc96ffd640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9815ca60 0x7fcc981535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.749 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc96ffd640 1 -- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc9815ce40 msgr2=0x7fcc981f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:00.750 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc96ffd640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc9815ce40 0x7fcc981f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.750 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc96ffd640 1 -- 172.21.15.32:0/3353539874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88009470 con 0x7fcc9806b980 2023-10-16T08:16:00.750 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc96ffd640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 0x7fcc98153070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc880ac040 tx=0x7fcc8801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:00.750 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc880aa670 con 0x7fcc9806b980 2023-10-16T08:16:00.750 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc880bd610 con 0x7fcc9806b980 2023-10-16T08:16:00.751 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc980c6a70 con 0x7fcc9806b980 2023-10-16T08:16:00.751 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc880bd9c0 con 0x7fcc9806b980 2023-10-16T08:16:00.751 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc980c7030 con 0x7fcc9806b980 2023-10-16T08:16:00.751 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcc880b33f0 con 0x7fcc9806b980 2023-10-16T08:16:00.751 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fcc880bde20 con 0x7fcc9806b980 2023-10-16T08:16:00.751 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9804e170 con 0x7fcc9806b980 2023-10-16T08:16:00.752 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.744+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fcc880c3060 con 0x7fcc9806b980 2023-10-16T08:16:00.785 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-16T08:16:00.785 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-16T08:16:00.785 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-10-16T08:16:00.806 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc9806b220 con 0x7fcc9806b980 2023-10-16T08:16:00.807 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc7ffff640 1 -- 172.21.15.32:0/3353539874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fcc880b3700 con 0x7fcc9806b980 2023-10-16T08:16:00.807 INFO:teuthology.misc.health.smithi032.stdout: 2023-10-16T08:16:00.807 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:15:59.468950+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:00.808 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 msgr2=0x7fcc98153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:00.809 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 0x7fcc98153070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc880ac040 tx=0x7fcc8801ad30 comp rx=0 tx=0).stop 2023-10-16T08:16:00.809 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 shutdown_connections 2023-10-16T08:16:00.809 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc9815ce40 0x7fcc981f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.809 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcc9815ca60 0x7fcc981535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.810 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 --2- 172.21.15.32:0/3353539874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc9806b980 0x7fcc98153070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:00.810 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 >> 172.21.15.32:0/3353539874 conn(0x7fcc9808c350 msgr2=0x7fcc9815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:00.810 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 shutdown_connections 2023-10-16T08:16:00.810 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:00.804+0000 7fcc9d886640 1 -- 172.21.15.32:0/3353539874 wait complete. 2023-10-16T08:16:00.825 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T08:16:00.828 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:00.824+0000 7ff82880c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T08:16:00.971 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:00.968+0000 7ff82880c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:16:01.161 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:01.160+0000 7ff82880c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T08:16:01.237 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:01.236+0000 7ff82880c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T08:16:01.239 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:01.237+0000 7f4fc5d63180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T08:16:01.324 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:01.321+0000 7f4fc5d63180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T08:16:01.382 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:01.380+0000 7ff82880c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:16:01.515 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:01.513+0000 7f4fc5d63180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T08:16:01.598 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:01.597+0000 7f4fc5d63180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T08:16:01.925 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:01.921+0000 7f4fc5d63180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T08:16:01.993 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:01.989+0000 7f4fc5d63180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T08:16:02.091 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:02.088+0000 7ff82880c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T08:16:02.183 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:02.180+0000 7ff82880c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T08:16:02.383 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:02.380+0000 7ff82880c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T08:16:02.473 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:02.468+0000 7ff82880c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T08:16:02.823 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:02.820+0000 7ff82880c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T08:16:02.894 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-10-16T08:16:02.892+0000 7ff82880c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T08:16:03.021 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:03.017+0000 7f4f6d4c7640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-16T08:16:03.022 INFO:tasks.ceph.mgr.x.smithi186.stderr:2023-10-16T08:16:03.021+0000 7f4f6d4c7640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-16T08:16:03.903 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-10-16T08:16:03.900+0000 7f9ab1c4f640 -1 osd.2 0 log_to_monitors true 2023-10-16T08:16:03.907 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-10-16T08:16:03.904+0000 7f70fdc57640 -1 osd.0 0 log_to_monitors true 2023-10-16T08:16:03.930 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-10-16T08:16:03.928+0000 7f0914507640 -1 osd.1 0 log_to_monitors true 2023-10-16T08:16:04.041 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-10-16T08:16:04.037+0000 7f325d28e640 -1 osd.5 0 log_to_monitors true 2023-10-16T08:16:04.179 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-10-16T08:16:04.177+0000 7fd3169ad640 -1 osd.7 0 log_to_monitors true 2023-10-16T08:16:04.184 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-10-16T08:16:04.180+0000 7f94feaac640 -1 osd.3 0 log_to_monitors true 2023-10-16T08:16:04.206 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-10-16T08:16:04.205+0000 7f9bc7c29640 -1 osd.6 0 log_to_monitors true 2023-10-16T08:16:04.220 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-10-16T08:16:04.217+0000 7f7508b53640 -1 osd.4 0 log_to_monitors true 2023-10-16T08:16:07.480 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-10-16T08:16:07.475+0000 7f09104af640 -1 osd.1 0 waiting for initial osdmap 2023-10-16T08:16:07.508 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-10-16T08:16:07.503+0000 7f70f9bff640 -1 osd.0 0 waiting for initial osdmap 2023-10-16T08:16:07.522 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-10-16T08:16:07.519+0000 7f94faa54640 -1 osd.3 0 waiting for initial osdmap 2023-10-16T08:16:07.534 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-10-16T08:16:07.531+0000 7f09082e4640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:07.541 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-10-16T08:16:07.539+0000 7f70f1a34640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:07.562 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-10-16T08:16:07.559+0000 7f94f2889640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:07.632 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:07.637 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-10-16T08:16:07.631+0000 7f9aadbf7640 -1 osd.2 0 waiting for initial osdmap 2023-10-16T08:16:07.657 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-10-16T08:16:07.655+0000 7f9aa5a2c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:07.765 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.759+0000 7fd29ed2a640 1 Processor -- start 2023-10-16T08:16:07.765 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.759+0000 7fd29ed2a640 1 -- start start 2023-10-16T08:16:07.766 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29ed2a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd2981601b0 0x7fd2980cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:07.767 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29ed2a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980cdce0 con 0x7fd2980cbf50 2023-10-16T08:16:07.767 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29ed2a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980cdf10 con 0x7fd2980b1810 2023-10-16T08:16:07.767 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29ed2a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2980cfcf0 con 0x7fd2981601b0 2023-10-16T08:16:07.767 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29d2c7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd2980cbf50 0x7fd29806afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50916/0 (socket says 172.21.15.32:50916) 2023-10-16T08:16:07.767 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29d2c7640 1 -- 172.21.15.32:0/2410666136 learned_addr learned my addr 172.21.15.32:0/2410666136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:07.768 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29cac6640 1 --2- 172.21.15.32:0/2410666136 >> v2:172.21.15.186:3300/0 conn(0x7fd2981601b0 0x7fd2980cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:07.768 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd2977fe640 1 -- 172.21.15.32:0/2410666136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039826484 0 0) 0x7fd2980cdce0 con 0x7fd2980cbf50 2023-10-16T08:16:07.768 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd2977fe640 1 -- 172.21.15.32:0/2410666136 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd288003530 con 0x7fd2980cbf50 2023-10-16T08:16:07.769 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29cac6640 1 -- 172.21.15.32:0/2410666136 >> v1:172.21.15.186:6789/0 conn(0x7fd2980b1810 legacy=0x7fd2980cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.769 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29cac6640 1 -- 172.21.15.32:0/2410666136 >> v1:172.21.15.32:6789/0 conn(0x7fd2980cbf50 legacy=0x7fd29806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.769 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.763+0000 7fd29cac6640 1 -- 172.21.15.32:0/2410666136 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980cf630 con 0x7fd2981601b0 2023-10-16T08:16:07.772 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.767+0000 7fd29cac6640 1 --2- 172.21.15.32:0/2410666136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd2980cd090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0097c0 tx=0x7fd28c01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c9f11a62c2023a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:07.773 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.767+0000 7fd2977fe640 1 -- 172.21.15.32:0/2410666136 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0ab600 con 0x7fd2981601b0 2023-10-16T08:16:07.773 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.767+0000 7fd2977fe640 1 -- 172.21.15.32:0/2410666136 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c0a7050 con 0x7fd2981601b0 2023-10-16T08:16:07.773 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.767+0000 7fd2977fe640 1 -- 172.21.15.32:0/2410666136 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0ab980 con 0x7fd2981601b0 2023-10-16T08:16:07.774 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2410666136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 msgr2=0x7fd2980cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.775 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2410666136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd2980cd090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0097c0 tx=0x7fd28c01c360 comp rx=0 tx=0).stop 2023-10-16T08:16:07.775 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2410666136 shutdown_connections 2023-10-16T08:16:07.775 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2410666136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd2980cd090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.775 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2410666136 >> 172.21.15.32:0/2410666136 conn(0x7fd2980d0810 msgr2=0x7fd2980d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:07.775 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2410666136 shutdown_connections 2023-10-16T08:16:07.776 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2410666136 wait complete. 2023-10-16T08:16:07.778 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 Processor -- start 2023-10-16T08:16:07.778 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- start start 2023-10-16T08:16:07.778 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 0x7fd29814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:07.778 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2980cbf50 0x7fd29814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:07.779 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd298157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:07.779 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298150270 con 0x7fd2980b1810 2023-10-16T08:16:07.779 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298151420 con 0x7fd2981601b0 2023-10-16T08:16:07.779 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29ed2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981566a0 con 0x7fd2980cbf50 2023-10-16T08:16:07.779 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29cac6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 0x7fd29814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:07.780 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29cac6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 0x7fd29814bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41460/0 (socket says 172.21.15.32:41460) 2023-10-16T08:16:07.780 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd29cac6640 1 -- 172.21.15.32:0/2426913838 learned_addr learned my addr 172.21.15.32:0/2426913838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:07.780 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.771+0000 7fd297fff640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2980cbf50 0x7fd29814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:07.780 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29d2c7640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd298157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:07.780 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29cac6640 1 -- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2980cbf50 msgr2=0x7fd29814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.780 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29cac6640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2980cbf50 0x7fd29814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.781 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29cac6640 1 -- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 msgr2=0x7fd298157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.781 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29cac6640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd298157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.781 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29cac6640 1 -- 172.21.15.32:0/2426913838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009420 con 0x7fd2980b1810 2023-10-16T08:16:07.781 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29cac6640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 0x7fd29814bab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd28c019f90 tx=0x7fd28c019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:07.781 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0abdb0 con 0x7fd2980b1810 2023-10-16T08:16:07.782 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c0b4400 con 0x7fd2980b1810 2023-10-16T08:16:07.782 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0b47b0 con 0x7fd2980b1810 2023-10-16T08:16:07.782 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298156980 con 0x7fd2980b1810 2023-10-16T08:16:07.782 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981510b0 con 0x7fd2980b1810 2023-10-16T08:16:07.783 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298156500 con 0x7fd2980b1810 2023-10-16T08:16:07.784 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 85794+0+0 (secure 0 0 0) 0x7fd28c0b2040 con 0x7fd2980b1810 2023-10-16T08:16:07.784 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd2957fa640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd27c066410 0x7fd27c0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:07.784 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.775+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd28c11ce20 con 0x7fd2980b1810 2023-10-16T08:16:07.784 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.779+0000 7fd297fff640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd27c066410 0x7fd27c0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:07.784 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.779+0000 7fd297fff640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd27c066410 0x7fd27c0688d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd298158f10 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:07.786 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.783+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd298156500 con 0x7fd2980b1810 2023-10-16T08:16:07.945 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.939+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd298156500 con 0x7fd2980b1810 2023-10-16T08:16:07.945 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd2957fa640 1 -- 172.21.15.32:0/2426913838 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd298156500 con 0x7fd2980b1810 2023-10-16T08:16:07.946 INFO:teuthology.misc.health.smithi032.stdout: 2023-10-16T08:16:07.946 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":12,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:05.721684+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:07.947 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd27c066410 msgr2=0x7fd27c0688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.948 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd27c066410 0x7fd27c0688d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd298158f10 tx=0x7fd284009040 comp rx=0 tx=0).stop 2023-10-16T08:16:07.948 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 msgr2=0x7fd29814bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:07.948 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 0x7fd29814bab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd28c019f90 tx=0x7fd28c019100 comp rx=0 tx=0).stop 2023-10-16T08:16:07.948 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 shutdown_connections 2023-10-16T08:16:07.948 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2981601b0 0x7fd298157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.949 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd27c066410 0x7fd27c0688d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.949 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd2980cbf50 0x7fd29814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.949 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 --2- 172.21.15.32:0/2426913838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2980b1810 0x7fd29814bab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:07.949 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 >> 172.21.15.32:0/2426913838 conn(0x7fd2980d0810 msgr2=0x7fd298159180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:07.949 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 shutdown_connections 2023-10-16T08:16:07.949 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:07.943+0000 7fd29ed2a640 1 -- 172.21.15.32:0/2426913838 wait complete. 2023-10-16T08:16:07.965 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T08:16:08.168 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-10-16T08:16:08.165+0000 7f7504afb640 -1 osd.4 0 waiting for initial osdmap 2023-10-16T08:16:08.176 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-10-16T08:16:08.173+0000 7f9bc43e5640 -1 osd.6 0 waiting for initial osdmap 2023-10-16T08:16:08.198 INFO:tasks.ceph.osd.4.smithi186.stderr:2023-10-16T08:16:08.197+0000 7f74fc930640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:08.207 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-10-16T08:16:08.205+0000 7fd312955640 -1 osd.7 0 waiting for initial osdmap 2023-10-16T08:16:08.218 INFO:tasks.ceph.osd.6.smithi186.stderr:2023-10-16T08:16:08.217+0000 7f9bbba06640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:08.232 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-10-16T08:16:08.229+0000 7f3259236640 -1 osd.5 0 waiting for initial osdmap 2023-10-16T08:16:08.262 INFO:tasks.ceph.osd.7.smithi186.stderr:2023-10-16T08:16:08.261+0000 7fd30a78a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:08.286 INFO:tasks.ceph.osd.5.smithi186.stderr:2023-10-16T08:16:08.285+0000 7f325106b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:16:14.771 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:14.918 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec3db72640 1 Processor -- start 2023-10-16T08:16:14.919 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec3db72640 1 -- start start 2023-10-16T08:16:14.920 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec3db72640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fec3815ce40 0x7fec380ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:14.921 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec3db72640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380cfd50 con 0x7fec380ce540 2023-10-16T08:16:14.921 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec3db72640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380d0030 con 0x7fec3815ca60 2023-10-16T08:16:14.921 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec3db72640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec380cd520 con 0x7fec3815ce40 2023-10-16T08:16:14.921 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec37fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec380ce540 0x7fec380d0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43248/0 (socket says 172.21.15.32:43248) 2023-10-16T08:16:14.921 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec37fff640 1 -- 172.21.15.32:0/4262894943 learned_addr learned my addr 172.21.15.32:0/4262894943 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:14.921 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec36ffd640 1 --2- 172.21.15.32:0/4262894943 >> v2:172.21.15.186:3300/0 conn(0x7fec3815ce40 0x7fec380ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:14.922 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195953185 0 0) 0x7fec380cfd50 con 0x7fec380ce540 2023-10-16T08:16:14.923 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec20003530 con 0x7fec380ce540 2023-10-16T08:16:14.923 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447782124 0 0) 0x7fec380d0030 con 0x7fec3815ca60 2023-10-16T08:16:14.923 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec380cfd50 con 0x7fec3815ca60 2023-10-16T08:16:14.923 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec36ffd640 1 -- 172.21.15.32:0/4262894943 >> v1:172.21.15.186:6789/0 conn(0x7fec3815ca60 legacy=0x7fec3806b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:14.923 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.915+0000 7fec36ffd640 1 -- 172.21.15.32:0/4262894943 >> v1:172.21.15.32:6789/0 conn(0x7fec380ce540 legacy=0x7fec380d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:14.924 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec36ffd640 1 -- 172.21.15.32:0/4262894943 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380cd800 con 0x7fec3815ce40 2023-10-16T08:16:14.924 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec36ffd640 1 --2- 172.21.15.32:0/4262894943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec3815ce40 0x7fec380ce000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0097c0 tx=0x7fec2c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a2abd028f4765d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:14.925 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0ac640 con 0x7fec3815ce40 2023-10-16T08:16:14.925 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c0ac7a0 con 0x7fec3815ce40 2023-10-16T08:16:14.925 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec367fc640 1 -- 172.21.15.32:0/4262894943 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0acb30 con 0x7fec3815ce40 2023-10-16T08:16:14.926 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec3db72640 1 -- 172.21.15.32:0/4262894943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec3815ce40 msgr2=0x7fec380ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:14.926 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec3db72640 1 --2- 172.21.15.32:0/4262894943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec3815ce40 0x7fec380ce000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0097c0 tx=0x7fec2c01c480 comp rx=0 tx=0).stop 2023-10-16T08:16:14.926 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec3db72640 1 -- 172.21.15.32:0/4262894943 shutdown_connections 2023-10-16T08:16:14.927 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec3db72640 1 --2- 172.21.15.32:0/4262894943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec3815ce40 0x7fec380ce000 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:14.927 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.919+0000 7fec3db72640 1 -- 172.21.15.32:0/4262894943 >> 172.21.15.32:0/4262894943 conn(0x7fec3808c350 msgr2=0x7fec3809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:14.927 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 -- 172.21.15.32:0/4262894943 shutdown_connections 2023-10-16T08:16:14.927 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 -- 172.21.15.32:0/4262894943 wait complete. 2023-10-16T08:16:14.928 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 Processor -- start 2023-10-16T08:16:14.929 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 -- start start 2023-10-16T08:16:14.929 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec380ce540 0x7fec3815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:14.929 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 0x7fec38154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:14.929 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec3815ce40 0x7fec380c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:14.929 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c7210 con 0x7fec3815ca60 2023-10-16T08:16:14.929 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec38154d30 con 0x7fec380ce540 2023-10-16T08:16:14.930 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec3db72640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c7d30 con 0x7fec3815ce40 2023-10-16T08:16:14.930 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 0x7fec38154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:14.930 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 0x7fec38154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41084/0 (socket says 172.21.15.32:41084) 2023-10-16T08:16:14.930 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 -- 172.21.15.32:0/3513585573 learned_addr learned my addr 172.21.15.32:0/3513585573 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:14.930 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec37fff640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec3815ce40 0x7fec380c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:14.931 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec377fe640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec380ce540 0x7fec3815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:14.932 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 -- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec3815ce40 msgr2=0x7fec380c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:14.932 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec3815ce40 0x7fec380c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:14.932 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 -- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec380ce540 msgr2=0x7fec3815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:14.932 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec380ce540 0x7fec3815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:14.933 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 -- 172.21.15.32:0/3513585573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c009420 con 0x7fec3815ca60 2023-10-16T08:16:14.933 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.923+0000 7fec36ffd640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 0x7fec38154140 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0097c0 tx=0x7fec2c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:14.933 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0b5600 con 0x7fec3815ca60 2023-10-16T08:16:14.933 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c0b5780 con 0x7fec3815ca60 2023-10-16T08:16:14.933 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3815d9f0 con 0x7fec3815ca60 2023-10-16T08:16:14.934 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0b5b30 con 0x7fec3815ca60 2023-10-16T08:16:14.934 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380c76e0 con 0x7fec3815ca60 2023-10-16T08:16:14.934 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380c7ea0 con 0x7fec3815ca60 2023-10-16T08:16:14.934 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fec2c0b3040 con 0x7fec3815ca60 2023-10-16T08:16:14.935 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec1ffff640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec240664a0 0x7fec24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:14.935 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.927+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec2c11dc40 con 0x7fec3815ca60 2023-10-16T08:16:14.935 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.931+0000 7fec377fe640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec240664a0 0x7fec24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:14.936 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.931+0000 7fec377fe640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec240664a0 0x7fec24068960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec28001240 tx=0x7fec28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:14.938 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:14.935+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fec2c0eb420 con 0x7fec3815ca60 2023-10-16T08:16:15.098 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec3804e170 con 0x7fec3815ca60 2023-10-16T08:16:15.099 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec1ffff640 1 -- 172.21.15.32:0/3513585573 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7fec2c0eea40 con 0x7fec3815ca60 2023-10-16T08:16:15.099 INFO:teuthology.misc.health.smithi032.stdout: 2023-10-16T08:16:15.100 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":17,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:12.873979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:15.101 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec240664a0 msgr2=0x7fec24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.101 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec240664a0 0x7fec24068960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec28001240 tx=0x7fec28009040 comp rx=0 tx=0).stop 2023-10-16T08:16:15.102 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 msgr2=0x7fec38154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.102 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 0x7fec38154140 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0097c0 tx=0x7fec2c019220 comp rx=0 tx=0).stop 2023-10-16T08:16:15.102 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 shutdown_connections 2023-10-16T08:16:15.102 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec240664a0 0x7fec24068960 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.103 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec3815ce40 0x7fec380c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.103 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec3815ca60 0x7fec38154140 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.103 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 --2- 172.21.15.32:0/3513585573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec380ce540 0x7fec3815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.103 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.095+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 >> 172.21.15.32:0/3513585573 conn(0x7fec3808c350 msgr2=0x7fec38157780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:15.103 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.099+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 shutdown_connections 2023-10-16T08:16:15.104 INFO:teuthology.misc.health.smithi032.stderr:2023-10-16T08:16:15.099+0000 7fec3db72640 1 -- 172.21.15.32:0/3513585573 wait complete. 2023-10-16T08:16:15.119 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-16T08:16:15.119 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-16T08:16:15.119 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T08:16:15.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T08:16:15.123 INFO:tasks.ceph.mds.a:Started 2023-10-16T08:16:15.124 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T08:16:15.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T08:16:15.127 INFO:tasks.ceph.mds.c:Started 2023-10-16T08:16:15.128 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T08:16:15.128 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T08:16:15.132 INFO:tasks.ceph.mds.b:Started 2023-10-16T08:16:15.132 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T08:16:15.133 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T08:16:15.136 INFO:tasks.ceph.mds.d:Started 2023-10-16T08:16:15.137 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-16T08:16:15.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:15.235 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-10-16T08:16:15.236 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-10-16T08:16:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.283+0000 7f14f78d6640 1 Processor -- start 2023-10-16T08:16:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.283+0000 7f14f78d6640 1 -- start start 2023-10-16T08:16:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f78d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14f015cb90 0x7f14f00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f78d6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f14f00d15d0 0x7f14f00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f78d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f00ce770 con 0x7f14f00d3010 2023-10-16T08:16:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f78d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00d0600 con 0x7f14f015cb90 2023-10-16T08:16:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f78d6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00c6450 con 0x7f14f00d15d0 2023-10-16T08:16:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5672640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14f015cb90 0x7f14f00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5e73640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f14f00d15d0 0x7f14f00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f4e71640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14f00d3010 0x7f14f00d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43264/0 (socket says 172.21.15.32:43264) 2023-10-16T08:16:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5672640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14f015cb90 0x7f14f00cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41096/0 (socket says 172.21.15.32:41096) 2023-10-16T08:16:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f4e71640 1 -- 172.21.15.32:0/1788093618 learned_addr learned my addr 172.21.15.32:0/1788093618 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:15.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5672640 1 -- 172.21.15.32:0/1788093618 >> v2:172.21.15.186:3300/0 conn(0x7f14f00d15d0 msgr2=0x7f14f00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5672640 1 --2- 172.21.15.32:0/1788093618 >> v2:172.21.15.186:3300/0 conn(0x7f14f00d15d0 0x7f14f00c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5672640 1 -- 172.21.15.32:0/1788093618 >> v1:172.21.15.32:6789/0 conn(0x7f14f00d3010 legacy=0x7f14f00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.287+0000 7f14f5672640 1 -- 172.21.15.32:0/1788093618 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00d08a0 con 0x7f14f015cb90 2023-10-16T08:16:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.291+0000 7f14f5672640 1 --2- 172.21.15.32:0/1788093618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f015cb90 0x7f14f00cfb10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14f00ccee0 tx=0x7f14ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5ec91c72531cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14e7fff640 1 -- 172.21.15.32:0/1788093618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0ad680 con 0x7f14f015cb90 2023-10-16T08:16:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14e7fff640 1 -- 172.21.15.32:0/1788093618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec0ad7e0 con 0x7f14f015cb90 2023-10-16T08:16:15.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14e7fff640 1 -- 172.21.15.32:0/1788093618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0adb70 con 0x7f14f015cb90 2023-10-16T08:16:15.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 -- 172.21.15.32:0/1788093618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f015cb90 msgr2=0x7f14f00cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 --2- 172.21.15.32:0/1788093618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f015cb90 0x7f14f00cfb10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14f00ccee0 tx=0x7f14ec01c380 comp rx=0 tx=0).stop 2023-10-16T08:16:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 -- 172.21.15.32:0/1788093618 shutdown_connections 2023-10-16T08:16:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 --2- 172.21.15.32:0/1788093618 >> v2:172.21.15.186:3300/0 conn(0x7f14f00d15d0 0x7f14f00c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 --2- 172.21.15.32:0/1788093618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f015cb90 0x7f14f00cfb10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 -- 172.21.15.32:0/1788093618 >> 172.21.15.32:0/1788093618 conn(0x7f14f009bc50 msgr2=0x7f14f00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 -- 172.21.15.32:0/1788093618 shutdown_connections 2023-10-16T08:16:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.323+0000 7f14f78d6640 1 -- 172.21.15.32:0/1788093618 wait complete. 2023-10-16T08:16:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 Processor -- start 2023-10-16T08:16:15.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 -- start start 2023-10-16T08:16:15.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14f00d15d0 0x7f14f014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 0x7f14f00b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14f015cb90 0x7f14f01500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b7700 con 0x7f14f00d3010 2023-10-16T08:16:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b6ac0 con 0x7f14f015cb90 2023-10-16T08:16:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f78d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b6d60 con 0x7f14f00d15d0 2023-10-16T08:16:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 0x7f14f00b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f5672640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14f00d15d0 0x7f14f014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 0x7f14f00b7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41110/0 (socket says 172.21.15.32:41110) 2023-10-16T08:16:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f5672640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14f00d15d0 0x7f14f014f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:43042/0 (socket says 172.21.15.32:43042) 2023-10-16T08:16:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 -- 172.21.15.32:0/20815200 learned_addr learned my addr 172.21.15.32:0/20815200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:15.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f5e73640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14f015cb90 0x7f14f01500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 -- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14f00d15d0 msgr2=0x7f14f014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14f00d15d0 0x7f14f014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 -- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14f015cb90 msgr2=0x7f14f01500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14f015cb90 0x7f14f01500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 -- 172.21.15.32:0/20815200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0093b0 con 0x7f14f00d3010 2023-10-16T08:16:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14f4e71640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 0x7f14f00b7b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005890 tx=0x7f14e000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.327+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0005cf0 con 0x7f14f00d3010 2023-10-16T08:16:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e0005e50 con 0x7f14f00d3010 2023-10-16T08:16:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f00b7040 con 0x7f14f00d3010 2023-10-16T08:16:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e00072b0 con 0x7f14f00d3010 2023-10-16T08:16:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f014fe50 con 0x7f14f00d3010 2023-10-16T08:16:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f00b7840 con 0x7f14f00d3010 2023-10-16T08:16:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f14e0007410 con 0x7f14f00d3010 2023-10-16T08:16:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14e5ffb640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14d40664d0 0x7f14d4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14e0082f90 con 0x7f14f00d3010 2023-10-16T08:16:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.331+0000 7f14f5672640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14d40664d0 0x7f14d4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.335+0000 7f14f5672640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14d40664d0 0x7f14d4068990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0af000 tx=0x7f14ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.339+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14f00b7840 con 0x7f14f00d3010 2023-10-16T08:16:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.495+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f00b7840 con 0x7f14f00d3010 2023-10-16T08:16:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14e5ffb640 1 -- 172.21.15.32:0/20815200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f14f00b7840 con 0x7f14f00d3010 2023-10-16T08:16:15.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:15.503 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:12.873979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14d40664d0 msgr2=0x7f14d4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14d40664d0 0x7f14d4068990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0af000 tx=0x7f14ec0a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 msgr2=0x7f14f00b7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 0x7f14f00b7b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005890 tx=0x7f14e000c960 comp rx=0 tx=0).stop 2023-10-16T08:16:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 shutdown_connections 2023-10-16T08:16:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14f015cb90 0x7f14f01500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14f00d3010 0x7f14f00b7b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14d40664d0 0x7f14d4068990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 --2- 172.21.15.32:0/20815200 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14f00d15d0 0x7f14f014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 >> 172.21.15.32:0/20815200 conn(0x7f14f009bc50 msgr2=0x7f14f00d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 shutdown_connections 2023-10-16T08:16:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.499+0000 7f14f78d6640 1 -- 172.21.15.32:0/20815200 wait complete. 2023-10-16T08:16:15.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:16:15.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:16:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 Processor -- start 2023-10-16T08:16:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 -- start start 2023-10-16T08:16:15.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f85bc0d0210 0x7f85bc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85bc0b84e0 0x7f85bc0b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0d1670 con 0x7f85bc15cba0 2023-10-16T08:16:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0d1460 con 0x7f85bc0b84e0 2023-10-16T08:16:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85c0846640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0d1950 con 0x7f85bc0d0210 2023-10-16T08:16:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85bc0b84e0 0x7f85bc0b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85bc0b84e0 0x7f85bc0b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41126/0 (socket says 172.21.15.32:41126) 2023-10-16T08:16:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 -- 172.21.15.32:0/1416764562 learned_addr learned my addr 172.21.15.32:0/1416764562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85b3fff640 1 --2- 172.21.15.32:0/1416764562 >> v2:172.21.15.186:3300/0 conn(0x7f85bc0d0210 0x7f85bc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 -- 172.21.15.32:0/1416764562 >> v2:172.21.15.186:3300/0 conn(0x7f85bc0d0210 msgr2=0x7f85bc0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 --2- 172.21.15.32:0/1416764562 >> v2:172.21.15.186:3300/0 conn(0x7f85bc0d0210 0x7f85bc0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 -- 172.21.15.32:0/1416764562 >> v1:172.21.15.186:6789/0 conn(0x7f85bc15cba0 legacy=0x7f85bc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.667+0000 7f85bad9d640 1 -- 172.21.15.32:0/1416764562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0d1c30 con 0x7f85bc0b84e0 2023-10-16T08:16:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85bad9d640 1 --2- 172.21.15.32:0/1416764562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0b84e0 0x7f85bc0b5cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0cf010 tx=0x7f85b401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69688e7bf9245836 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:15.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85b9d9b640 1 -- 172.21.15.32:0/1416764562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b40a5800 con 0x7f85bc0b84e0 2023-10-16T08:16:15.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85b9d9b640 1 -- 172.21.15.32:0/1416764562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b40a5960 con 0x7f85bc0b84e0 2023-10-16T08:16:15.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85b9d9b640 1 -- 172.21.15.32:0/1416764562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b40a5cf0 con 0x7f85bc0b84e0 2023-10-16T08:16:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 -- 172.21.15.32:0/1416764562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0b84e0 msgr2=0x7f85bc0b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 --2- 172.21.15.32:0/1416764562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0b84e0 0x7f85bc0b5cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0cf010 tx=0x7f85b401c940 comp rx=0 tx=0).stop 2023-10-16T08:16:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 -- 172.21.15.32:0/1416764562 shutdown_connections 2023-10-16T08:16:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 --2- 172.21.15.32:0/1416764562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0b84e0 0x7f85bc0b5cf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 --2- 172.21.15.32:0/1416764562 >> v2:172.21.15.186:3300/0 conn(0x7f85bc0d0210 0x7f85bc0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 -- 172.21.15.32:0/1416764562 >> 172.21.15.32:0/1416764562 conn(0x7f85bc0719c0 msgr2=0x7f85bc071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 -- 172.21.15.32:0/1416764562 shutdown_connections 2023-10-16T08:16:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.671+0000 7f85c0846640 1 -- 172.21.15.32:0/1416764562 wait complete. 2023-10-16T08:16:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 Processor -- start 2023-10-16T08:16:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- start start 2023-10-16T08:16:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85bc0b84e0 0x7f85bc148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 0x7f85bc148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85bc15cba0 0x7f85bc149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc148b50 con 0x7f85bc0d0210 2023-10-16T08:16:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0c6770 con 0x7f85bc0b84e0 2023-10-16T08:16:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc156f40 con 0x7f85bc15cba0 2023-10-16T08:16:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 0x7f85bc148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 0x7f85bc148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41130/0 (socket says 172.21.15.32:41130) 2023-10-16T08:16:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 -- 172.21.15.32:0/3315859135 learned_addr learned my addr 172.21.15.32:0/3315859135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85bad9d640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85bc15cba0 0x7f85bc149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85ba59c640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85bc0b84e0 0x7f85bc148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 -- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85bc15cba0 msgr2=0x7f85bc149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85bc15cba0 0x7f85bc149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 -- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85bc0b84e0 msgr2=0x7f85bc148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85bc0b84e0 0x7f85bc148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 -- 172.21.15.32:0/3315859135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4009b30 con 0x7f85bc0d0210 2023-10-16T08:16:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b3fff640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 0x7f85bc148cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e900 tx=0x7f85a400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4009690 con 0x7f85bc0d0210 2023-10-16T08:16:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400ca00 con 0x7f85bc0d0210 2023-10-16T08:16:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc1566a0 con 0x7f85bc0d0210 2023-10-16T08:16:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a400cd90 con 0x7f85bc0d0210 2023-10-16T08:16:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc156c60 con 0x7f85bc0d0210 2023-10-16T08:16:15.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.675+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc0b6030 con 0x7f85bc0d0210 2023-10-16T08:16:15.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.679+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f85a4013470 con 0x7f85bc0d0210 2023-10-16T08:16:15.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.679+0000 7f85b2ffd640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f859c0664a0 0x7f859c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:15.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.679+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85a4083de0 con 0x7f85bc0d0210 2023-10-16T08:16:15.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.679+0000 7f85ba59c640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f859c0664a0 0x7f859c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:15.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.679+0000 7f85ba59c640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f859c0664a0 0x7f859c068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f70 tx=0x7f85ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:15.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.683+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f85a40515a0 con 0x7f85bc0d0210 2023-10-16T08:16:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.839+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85bc04e170 con 0x7f85bc0d0210 2023-10-16T08:16:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.839+0000 7f85b2ffd640 1 -- 172.21.15.32:0/3315859135 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85a40170a0 con 0x7f85bc0d0210 2023-10-16T08:16:15.845 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:16:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f859c0664a0 msgr2=0x7f859c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f859c0664a0 0x7f859c068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000f70 tx=0x7f85ac009040 comp rx=0 tx=0).stop 2023-10-16T08:16:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 msgr2=0x7f85bc148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 0x7f85bc148cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85a400e900 tx=0x7f85a400edc0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 shutdown_connections 2023-10-16T08:16:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f859c0664a0 0x7f859c068960 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f85bc15cba0 0x7f85bc149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc0d0210 0x7f85bc148cd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 --2- 172.21.15.32:0/3315859135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85bc0b84e0 0x7f85bc148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 >> 172.21.15.32:0/3315859135 conn(0x7f85bc0719c0 msgr2=0x7f85bc153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 shutdown_connections 2023-10-16T08:16:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:15.843+0000 7f85c0846640 1 -- 172.21.15.32:0/3315859135 wait complete. 2023-10-16T08:16:15.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:15.983 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-10-16T08:16:16.000 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-10-16T08:16:16.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.027+0000 7f598c005640 1 Processor -- start 2023-10-16T08:16:16.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.027+0000 7f598c005640 1 -- start start 2023-10-16T08:16:16.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598c005640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598c005640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f59840c8cb0 0x7f59840cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598c005640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59840723d0 con 0x7f59841597c0 2023-10-16T08:16:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598c005640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59840717c0 con 0x7f59840c8cb0 2023-10-16T08:16:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598c005640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5984071aa0 con 0x7f5984159ba0 2023-10-16T08:16:16.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598a5a2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f59840c8cb0 0x7f59840cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:43078/0 (socket says 172.21.15.32:43078) 2023-10-16T08:16:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f598a5a2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f59840c8cb0 0x7f59840cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:45236/0 (socket says 172.21.15.32:45236) 2023-10-16T08:16:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 -- 172.21.15.32:0/958416849 learned_addr learned my addr 172.21.15.32:0/958416849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:16.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 -- 172.21.15.32:0/958416849 >> v2:172.21.15.186:3300/0 conn(0x7f59840c8cb0 msgr2=0x7f59840cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 --2- 172.21.15.32:0/958416849 >> v2:172.21.15.186:3300/0 conn(0x7f59840c8cb0 0x7f59840cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 -- 172.21.15.32:0/958416849 >> v1:172.21.15.186:6789/0 conn(0x7f59841597c0 legacy=0x7f59840ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:16.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.031+0000 7f59895a0640 1 -- 172.21.15.32:0/958416849 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984071d80 con 0x7f5984159ba0 2023-10-16T08:16:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f59895a0640 1 --2- 172.21.15.32:0/958416849 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840cae10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5974009870 tx=0x7f597401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afec795bdc77ea3d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f5988d9f640 1 -- 172.21.15.32:0/958416849 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740aa610 con 0x7f5984159ba0 2023-10-16T08:16:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f5988d9f640 1 -- 172.21.15.32:0/958416849 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740a6040 con 0x7f5984159ba0 2023-10-16T08:16:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f5988d9f640 1 -- 172.21.15.32:0/958416849 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740aa990 con 0x7f5984159ba0 2023-10-16T08:16:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- 172.21.15.32:0/958416849 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 msgr2=0x7f59840cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:16.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 --2- 172.21.15.32:0/958416849 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840cae10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5974009870 tx=0x7f597401c510 comp rx=0 tx=0).stop 2023-10-16T08:16:16.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- 172.21.15.32:0/958416849 shutdown_connections 2023-10-16T08:16:16.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 --2- 172.21.15.32:0/958416849 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840cae10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 --2- 172.21.15.32:0/958416849 >> v2:172.21.15.186:3300/0 conn(0x7f59840c8cb0 0x7f59840cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- 172.21.15.32:0/958416849 >> 172.21.15.32:0/958416849 conn(0x7f598406c190 msgr2=0x7f598406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- 172.21.15.32:0/958416849 shutdown_connections 2023-10-16T08:16:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- 172.21.15.32:0/958416849 wait complete. 2023-10-16T08:16:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 Processor -- start 2023-10-16T08:16:16.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- start start 2023-10-16T08:16:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 0x7f598414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59841597c0 0x7f59840c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:16.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984153840 con 0x7f59840c8cb0 2023-10-16T08:16:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984157d30 con 0x7f59841597c0 2023-10-16T08:16:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.035+0000 7f598c005640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5984158010 con 0x7f5984159ba0 2023-10-16T08:16:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 0x7f598414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f59895a0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59841597c0 0x7f59840c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 0x7f598414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41160/0 (socket says 172.21.15.32:41160) 2023-10-16T08:16:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 -- 172.21.15.32:0/1577117829 learned_addr learned my addr 172.21.15.32:0/1577117829 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f598a5a2640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 -- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 msgr2=0x7f59840c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 -- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59841597c0 msgr2=0x7f59840c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59841597c0 0x7f59840c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 -- 172.21.15.32:0/1577117829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740094d0 con 0x7f59840c8cb0 2023-10-16T08:16:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f5989da1640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 0x7f598414fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f598000c500 tx=0x7f598000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980005220 con 0x7f59840c8cb0 2023-10-16T08:16:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980005380 con 0x7f59840c8cb0 2023-10-16T08:16:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980007270 con 0x7f59840c8cb0 2023-10-16T08:16:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59841582f0 con 0x7f59840c8cb0 2023-10-16T08:16:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5984155e40 con 0x7f59840c8cb0 2023-10-16T08:16:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59840cd0d0 con 0x7f59840c8cb0 2023-10-16T08:16:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f59800073d0 con 0x7f59840c8cb0 2023-10-16T08:16:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.039+0000 7f597a7fc640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.043+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5980082e70 con 0x7f59840c8cb0 2023-10-16T08:16:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.043+0000 7f59895a0640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.043+0000 7f59895a0640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59740ae000 tx=0x7f59740a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.043+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5980050660 con 0x7f59840c8cb0 2023-10-16T08:16:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5984074010 con 0x7f59840c8cb0 2023-10-16T08:16:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f597a7fc640 1 -- 172.21.15.32:0/1577117829 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f5980053c80 con 0x7f59840c8cb0 2023-10-16T08:16:16.239 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:16.239 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-16T08:16:16.240 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2023-10-16T08:16:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 msgr2=0x7f5968068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59740ae000 tx=0x7f59740a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 msgr2=0x7f598414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 0x7f598414fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f598000c500 tx=0x7f598000c9c0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 shutdown_connections 2023-10-16T08:16:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5984159ba0 0x7f59840c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59841597c0 0x7f59840c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 --2- 172.21.15.32:0/1577117829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59840c8cb0 0x7f598414fc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:16.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 >> 172.21.15.32:0/1577117829 conn(0x7f598406c190 msgr2=0x7f598415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:16.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 shutdown_connections 2023-10-16T08:16:16.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:16.235+0000 7f598c005640 1 -- 172.21.15.32:0/1577117829 wait complete. 2023-10-16T08:16:16.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:16:16.593 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:16:18.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3f46b640 1 Processor -- start 2023-10-16T08:16:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3f46b640 1 -- start start 2023-10-16T08:16:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3f46b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 0x7fec380cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3f46b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380cfa30 con 0x7fec3815ca70 2023-10-16T08:16:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3f46b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380c65f0 con 0x7fec380c6ad0 2023-10-16T08:16:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3f46b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec380ce820 con 0x7fec380cf090 2023-10-16T08:16:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3d207640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec3815ca70 0x7fec380d3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43300/0 (socket says 172.21.15.32:43300) 2023-10-16T08:16:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3d207640 1 -- 172.21.15.32:0/807679193 learned_addr learned my addr 172.21.15.32:0/807679193 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3ca06640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 0x7fec380cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088817519 0 0) 0x7fec380cfa30 con 0x7fec3815ca70 2023-10-16T08:16:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec24003530 con 0x7fec3815ca70 2023-10-16T08:16:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209492611 0 0) 0x7fec380c65f0 con 0x7fec380c6ad0 2023-10-16T08:16:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec380cfa30 con 0x7fec380c6ad0 2023-10-16T08:16:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 284008333 0 0) 0x7fec24003530 con 0x7fec3815ca70 2023-10-16T08:16:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 msgr2=0x7fec380cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 0x7fec380cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 >> v1:172.21.15.186:6789/0 conn(0x7fec380c6ad0 legacy=0x7fec380c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380cdf80 con 0x7fec3815ca70 2023-10-16T08:16:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fec34001800 con 0x7fec3815ca70 2023-10-16T08:16:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 >> v1:172.21.15.32:6789/0 conn(0x7fec3815ca70 legacy=0x7fec380d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24003b40 0x7fec24003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec240047d0 0x7fec24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 0x7fec24005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380ce820 con 0x7fec24003b40 2023-10-16T08:16:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2400ba40 con 0x7fec240047d0 2023-10-16T08:16:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2400bc00 con 0x7fec380cf090 2023-10-16T08:16:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.411+0000 7fec3d207640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24003b40 0x7fec24003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3ca06640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec240047d0 0x7fec24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3da08640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 0x7fec24005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3d207640 1 -- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 msgr2=0x7fec24005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3d207640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec380cf090 0x7fec24005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3d207640 1 -- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec240047d0 msgr2=0x7fec24004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:18.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3d207640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec240047d0 0x7fec24004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3d207640 1 -- 172.21.15.32:0/807679193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24003530 con 0x7fec24003b40 2023-10-16T08:16:18.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3d207640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24003b40 0x7fec24003f40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec34007f90 tx=0x7fec3401cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6901bafe3d58745 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:18.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34007270 con 0x7fec24003b40 2023-10-16T08:16:18.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec340073d0 con 0x7fec24003b40 2023-10-16T08:16:18.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec2ffff640 1 -- 172.21.15.32:0/807679193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340057b0 con 0x7fec24003b40 2023-10-16T08:16:18.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3f46b640 1 -- 172.21.15.32:0/807679193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24003b40 msgr2=0x7fec24003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3f46b640 1 --2- 172.21.15.32:0/807679193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec24003b40 0x7fec24003f40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec34007f90 tx=0x7fec3401cad0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.415+0000 7fec3da08640 1 -- 172.21.15.32:0/807679193 reap_dead start 2023-10-16T08:16:18.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- 172.21.15.32:0/807679193 shutdown_connections 2023-10-16T08:16:18.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- 172.21.15.32:0/807679193 >> 172.21.15.32:0/807679193 conn(0x7fec3809bc50 msgr2=0x7fec380d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:18.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- 172.21.15.32:0/807679193 shutdown_connections 2023-10-16T08:16:18.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- 172.21.15.32:0/807679193 wait complete. 2023-10-16T08:16:18.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 Processor -- start 2023-10-16T08:16:18.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- start start 2023-10-16T08:16:18.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec24003b40 0x7fec381f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 0x7fec3815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38159270 0x7fec3815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381f5ff0 con 0x7fec38158df0 2023-10-16T08:16:18.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380b6d70 con 0x7fec38159270 2023-10-16T08:16:18.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3f46b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec380b7050 con 0x7fec24003b40 2023-10-16T08:16:18.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 0x7fec3815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 0x7fec3815f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41174/0 (socket says 172.21.15.32:41174) 2023-10-16T08:16:18.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 -- 172.21.15.32:0/217720706 learned_addr learned my addr 172.21.15.32:0/217720706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:18.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3da08640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38159270 0x7fec3815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3d207640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec24003b40 0x7fec381f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 -- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec24003b40 msgr2=0x7fec381f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec24003b40 0x7fec381f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 -- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38159270 msgr2=0x7fec3815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38159270 0x7fec3815fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.419+0000 7fec3ca06640 1 -- 172.21.15.32:0/217720706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380d38a0 con 0x7fec38158df0 2023-10-16T08:16:18.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec3ca06640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 0x7fec3815f650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fec28011960 tx=0x7fec28011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:18.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2800faf0 con 0x7fec38158df0 2023-10-16T08:16:18.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2800fc50 con 0x7fec38158df0 2023-10-16T08:16:18.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28014440 con 0x7fec38158df0 2023-10-16T08:16:18.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec380b7330 con 0x7fec38158df0 2023-10-16T08:16:18.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380b78f0 con 0x7fec38158df0 2023-10-16T08:16:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec38074010 con 0x7fec38158df0 2023-10-16T08:16:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fec2801d400 con 0x7fec38158df0 2023-10-16T08:16:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec2dffb640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec1c066520 0x7fec1c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec280842a0 con 0x7fec38158df0 2023-10-16T08:16:18.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.423+0000 7fec3d207640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec1c066520 0x7fec1c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.427+0000 7fec3d207640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec1c066520 0x7fec1c0689e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec340ab000 tx=0x7fec340a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:18.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.431+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fec28051b00 con 0x7fec38158df0 2023-10-16T08:16:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.619+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec380b87c0 con 0x7fec38158df0 2023-10-16T08:16:18.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec2dffb640 1 -- 172.21.15.32:0/217720706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7fec28055120 con 0x7fec38158df0 2023-10-16T08:16:18.626 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:18.627 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4311,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T08:16:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2023-10-16T08:16:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec1c066520 msgr2=0x7fec1c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec1c066520 0x7fec1c0689e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec340ab000 tx=0x7fec340a7070 comp rx=0 tx=0).stop 2023-10-16T08:16:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 msgr2=0x7fec3815f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 0x7fec3815f650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fec28011960 tx=0x7fec28011e20 comp rx=0 tx=0).stop 2023-10-16T08:16:18.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 shutdown_connections 2023-10-16T08:16:18.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec38159270 0x7fec3815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec38158df0 0x7fec3815f650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fec1c066520 0x7fec1c0689e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 --2- 172.21.15.32:0/217720706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fec24003b40 0x7fec381f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 >> 172.21.15.32:0/217720706 conn(0x7fec3809bc50 msgr2=0x7fec3806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 shutdown_connections 2023-10-16T08:16:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.623+0000 7fec3f46b640 1 -- 172.21.15.32:0/217720706 wait complete. 2023-10-16T08:16:18.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.783+0000 7f5e3efd0640 1 Processor -- start 2023-10-16T08:16:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 -- start start 2023-10-16T08:16:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e381597c0 0x7f5e380ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e380cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5e380adc10 0x7f5e380cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380723d0 con 0x7f5e381597c0 2023-10-16T08:16:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380717c0 con 0x7f5e380adc10 2023-10-16T08:16:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3efd0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38071aa0 con 0x7f5e38159ba0 2023-10-16T08:16:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e381597c0 0x7f5e380ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e381597c0 0x7f5e380ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41178/0 (socket says 172.21.15.32:41178) 2023-10-16T08:16:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 -- 172.21.15.32:0/868996131 learned_addr learned my addr 172.21.15.32:0/868996131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e37fff640 1 --2- 172.21.15.32:0/868996131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e380cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3d56d640 1 --2- 172.21.15.32:0/868996131 >> v2:172.21.15.186:3300/0 conn(0x7f5e380adc10 0x7f5e380cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 -- 172.21.15.32:0/868996131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 msgr2=0x7f5e380cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 --2- 172.21.15.32:0/868996131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e380cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 -- 172.21.15.32:0/868996131 >> v2:172.21.15.186:3300/0 conn(0x7f5e380adc10 msgr2=0x7f5e380cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 --2- 172.21.15.32:0/868996131 >> v2:172.21.15.186:3300/0 conn(0x7f5e380adc10 0x7f5e380cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.787+0000 7f5e3cd6c640 1 -- 172.21.15.32:0/868996131 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38071d80 con 0x7f5e381597c0 2023-10-16T08:16:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3cd6c640 1 --2- 172.21.15.32:0/868996131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e380ad6d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e3806e330 tx=0x7f5e2c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21d1d7e4285ba7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e377fe640 1 -- 172.21.15.32:0/868996131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0a7070 con 0x7f5e381597c0 2023-10-16T08:16:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e377fe640 1 -- 172.21.15.32:0/868996131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0ab430 con 0x7f5e381597c0 2023-10-16T08:16:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e377fe640 1 -- 172.21.15.32:0/868996131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0ab7c0 con 0x7f5e381597c0 2023-10-16T08:16:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 -- 172.21.15.32:0/868996131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 msgr2=0x7f5e380ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/868996131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e380ad6d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e3806e330 tx=0x7f5e2c01c4d0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 -- 172.21.15.32:0/868996131 shutdown_connections 2023-10-16T08:16:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/868996131 >> v2:172.21.15.186:3300/0 conn(0x7f5e380adc10 0x7f5e380cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/868996131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e380cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/868996131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e380ad6d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 -- 172.21.15.32:0/868996131 >> 172.21.15.32:0/868996131 conn(0x7f5e3806c190 msgr2=0x7f5e3806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 -- 172.21.15.32:0/868996131 shutdown_connections 2023-10-16T08:16:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 -- 172.21.15.32:0/868996131 wait complete. 2023-10-16T08:16:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 Processor -- start 2023-10-16T08:16:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.791+0000 7f5e3efd0640 1 -- start start 2023-10-16T08:16:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e380adc10 0x7f5e38154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e3814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e38156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38156b20 con 0x7f5e381597c0 2023-10-16T08:16:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380c8460 con 0x7f5e380adc10 2023-10-16T08:16:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38156460 con 0x7f5e38159ba0 2023-10-16T08:16:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e3814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e3814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41188/0 (socket says 172.21.15.32:41188) 2023-10-16T08:16:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 -- 172.21.15.32:0/1236933803 learned_addr learned my addr 172.21.15.32:0/1236933803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3cd6c640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e380adc10 0x7f5e38154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3d56d640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e38156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 -- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 msgr2=0x7f5e38156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e38156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 -- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e380adc10 msgr2=0x7f5e38154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e380adc10 0x7f5e38154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 -- 172.21.15.32:0/1236933803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0094d0 con 0x7f5e381597c0 2023-10-16T08:16:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3d56d640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e38156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e37fff640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e3814acb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400d5f0 tx=0x7f5e2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240125f0 con 0x7f5e381597c0 2023-10-16T08:16:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24012750 con 0x7f5e381597c0 2023-10-16T08:16:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24012ae0 con 0x7f5e381597c0 2023-10-16T08:16:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e38156740 con 0x7f5e381597c0 2023-10-16T08:16:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3814b1f0 con 0x7f5e381597c0 2023-10-16T08:16:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38074010 con 0x7f5e381597c0 2023-10-16T08:16:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.795+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5e24012c40 con 0x7f5e381597c0 2023-10-16T08:16:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.799+0000 7f5e357fa640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e1c0664a0 0x7f5e1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:18.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.799+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e24083f70 con 0x7f5e381597c0 2023-10-16T08:16:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.799+0000 7f5e3cd6c640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e1c0664a0 0x7f5e1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.799+0000 7f5e3cd6c640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e1c0664a0 0x7f5e1c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0af000 tx=0x7f5e2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:18.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.803+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5e24051730 con 0x7f5e381597c0 2023-10-16T08:16:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.963+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e380cd830 con 0x7f5e381597c0 2023-10-16T08:16:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.963+0000 7f5e357fa640 1 -- 172.21.15.32:0/1236933803 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f5e24054d50 con 0x7f5e381597c0 2023-10-16T08:16:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:18.969 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:12.873979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e1c0664a0 msgr2=0x7f5e1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e1c0664a0 0x7f5e1c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0af000 tx=0x7f5e2c008040 comp rx=0 tx=0).stop 2023-10-16T08:16:18.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 msgr2=0x7f5e3814acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:18.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e3814acb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400d5f0 tx=0x7f5e2400dab0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 shutdown_connections 2023-10-16T08:16:18.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e1c0664a0 0x7f5e1c068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e38159ba0 0x7f5e38156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e381597c0 0x7f5e3814acb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 --2- 172.21.15.32:0/1236933803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e380adc10 0x7f5e38154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 >> 172.21.15.32:0/1236933803 conn(0x7f5e3806c190 msgr2=0x7f5e380ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 shutdown_connections 2023-10-16T08:16:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:18.967+0000 7f5e3efd0640 1 -- 172.21.15.32:0/1236933803 wait complete. 2023-10-16T08:16:18.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:16:18.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:16:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 Processor -- start 2023-10-16T08:16:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 -- start start 2023-10-16T08:16:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88000d0210 0x7f88000d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 0x7f88000b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88000d1670 con 0x7f880015cba0 2023-10-16T08:16:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000d1460 con 0x7f88000d0210 2023-10-16T08:16:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f880679d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000d1950 con 0x7f88000b84e0 2023-10-16T08:16:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 0x7f88000b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f87ff7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f88000d0210 0x7f88000d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 0x7f88000b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:43140/0 (socket says 172.21.15.32:43140) 2023-10-16T08:16:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 -- 172.21.15.32:0/3392355028 learned_addr learned my addr 172.21.15.32:0/3392355028 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 -- 172.21.15.32:0/3392355028 >> v2:172.21.15.186:3300/0 conn(0x7f88000d0210 msgr2=0x7f88000d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 --2- 172.21.15.32:0/3392355028 >> v2:172.21.15.186:3300/0 conn(0x7f88000d0210 0x7f88000d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 -- 172.21.15.32:0/3392355028 >> v1:172.21.15.186:6789/0 conn(0x7f880015cba0 legacy=0x7f88000cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f8804d3a640 1 -- 172.21.15.32:0/3392355028 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000d1c30 con 0x7f88000b84e0 2023-10-16T08:16:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.131+0000 7f87feffd640 1 -- 172.21.15.32:0/3392355028 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131657277 0 0) 0x7f88000d1670 con 0x7f880015cba0 2023-10-16T08:16:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f8804d3a640 1 --2- 172.21.15.32:0/3392355028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 0x7f88000b5cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009e50 tx=0x7f87f801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548824d21a337802 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f87feffd640 1 -- 172.21.15.32:0/3392355028 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80a58e0 con 0x7f88000b84e0 2023-10-16T08:16:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f87feffd640 1 -- 172.21.15.32:0/3392355028 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80a5a40 con 0x7f88000b84e0 2023-10-16T08:16:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f87feffd640 1 -- 172.21.15.32:0/3392355028 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80a5dd0 con 0x7f88000b84e0 2023-10-16T08:16:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 -- 172.21.15.32:0/3392355028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 msgr2=0x7f88000b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 --2- 172.21.15.32:0/3392355028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 0x7f88000b5cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009e50 tx=0x7f87f801ca10 comp rx=0 tx=0).stop 2023-10-16T08:16:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 -- 172.21.15.32:0/3392355028 shutdown_connections 2023-10-16T08:16:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 --2- 172.21.15.32:0/3392355028 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000b84e0 0x7f88000b5cf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 --2- 172.21.15.32:0/3392355028 >> v2:172.21.15.186:3300/0 conn(0x7f88000d0210 0x7f88000d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 -- 172.21.15.32:0/3392355028 >> 172.21.15.32:0/3392355028 conn(0x7f88000719c0 msgr2=0x7f8800071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 -- 172.21.15.32:0/3392355028 shutdown_connections 2023-10-16T08:16:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.135+0000 7f880679d640 1 -- 172.21.15.32:0/3392355028 wait complete. 2023-10-16T08:16:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 Processor -- start 2023-10-16T08:16:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 -- start start 2023-10-16T08:16:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000b84e0 0x7f880014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000d0210 0x7f88000b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 0x7f88000b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8800072310 con 0x7f880015cba0 2023-10-16T08:16:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880014efd0 con 0x7f88000b84e0 2023-10-16T08:16:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f880679d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88001569f0 con 0x7f88000d0210 2023-10-16T08:16:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 0x7f88000b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 0x7f88000b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41202/0 (socket says 172.21.15.32:41202) 2023-10-16T08:16:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 -- 172.21.15.32:0/1282763716 learned_addr learned my addr 172.21.15.32:0/1282763716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f87fffff640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000b84e0 0x7f880014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f87ff7fe640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000d0210 0x7f88000b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 -- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000d0210 msgr2=0x7f88000b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000d0210 0x7f88000b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 -- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000b84e0 msgr2=0x7f880014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000b84e0 0x7f880014ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 -- 172.21.15.32:0/1282763716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8009b30 con 0x7f880015cba0 2023-10-16T08:16:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f8804d3a640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 0x7f88000b6f80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87f801a640 tx=0x7f87f80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.139+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b8850 con 0x7f880015cba0 2023-10-16T08:16:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80b89b0 con 0x7f880015cba0 2023-10-16T08:16:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b8d40 con 0x7f880015cba0 2023-10-16T08:16:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8800156cd0 con 0x7f880015cba0 2023-10-16T08:16:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880014e310 con 0x7f880015cba0 2023-10-16T08:16:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880004e170 con 0x7f880015cba0 2023-10-16T08:16:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87f80b6410 con 0x7f880015cba0 2023-10-16T08:16:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f87fcff9640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87ec0664a0 0x7f87ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87f811f1c0 con 0x7f880015cba0 2023-10-16T08:16:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.143+0000 7f87fffff640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87ec0664a0 0x7f87ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.147+0000 7f87fffff640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87ec0664a0 0x7f87ec068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87f0007fd0 tx=0x7f87f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.147+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f87f80ec9b0 con 0x7f880015cba0 2023-10-16T08:16:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f880014f440 con 0x7f880015cba0 2023-10-16T08:16:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f87fcff9640 1 -- 172.21.15.32:0/1282763716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87f80effd0 con 0x7f880015cba0 2023-10-16T08:16:19.311 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:16:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87ec0664a0 msgr2=0x7f87ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87ec0664a0 0x7f87ec068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87f0007fd0 tx=0x7f87f0009040 comp rx=0 tx=0).stop 2023-10-16T08:16:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 msgr2=0x7f88000b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 0x7f88000b6f80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87f801a640 tx=0x7f87f80197b0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 shutdown_connections 2023-10-16T08:16:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f880015cba0 0x7f88000b6f80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87ec0664a0 0x7f87ec068960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88000d0210 0x7f88000b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 --2- 172.21.15.32:0/1282763716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88000b84e0 0x7f880014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 >> 172.21.15.32:0/1282763716 conn(0x7f88000719c0 msgr2=0x7f88000b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 shutdown_connections 2023-10-16T08:16:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.307+0000 7f880679d640 1 -- 172.21.15.32:0/1282763716 wait complete. 2023-10-16T08:16:19.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T08:16:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 Processor -- start 2023-10-16T08:16:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 -- start start 2023-10-16T08:16:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfbc15cf00 0x7fbfbc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbfbc0cdee0 0x7fbfbc0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc0d1240 0x7fbfbc06ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc15d2e0 con 0x7fbfbc15cf00 2023-10-16T08:16:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0ce340 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc3ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0d16d0 con 0x7fbfbc0d1240 2023-10-16T08:16:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfbc15cf00 0x7fbfbc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfbc15cf00 0x7fbfbc0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41218/0 (socket says 172.21.15.32:41218) 2023-10-16T08:16:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 -- 172.21.15.32:0/472672568 learned_addr learned my addr 172.21.15.32:0/472672568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:19.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1071640 1 --2- 172.21.15.32:0/472672568 >> v2:172.21.15.186:3300/0 conn(0x7fbfbc0cdee0 0x7fbfbc0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc2073640 1 --2- 172.21.15.32:0/472672568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc0d1240 0x7fbfbc06ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 -- 172.21.15.32:0/472672568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc0d1240 msgr2=0x7fbfbc06ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 --2- 172.21.15.32:0/472672568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc0d1240 0x7fbfbc06ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 -- 172.21.15.32:0/472672568 >> v2:172.21.15.186:3300/0 conn(0x7fbfbc0cdee0 msgr2=0x7fbfbc0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 --2- 172.21.15.32:0/472672568 >> v2:172.21.15.186:3300/0 conn(0x7fbfbc0cdee0 0x7fbfbc0d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.471+0000 7fbfc1872640 1 -- 172.21.15.32:0/472672568 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0d1930 con 0x7fbfbc15cf00 2023-10-16T08:16:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc1872640 1 --2- 172.21.15.32:0/472672568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc15cf00 0x7fbfbc0cd9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0b5490 tx=0x7fbfb801c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e87d129d34c5bb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc0870640 1 -- 172.21.15.32:0/472672568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80ab400 con 0x7fbfbc15cf00 2023-10-16T08:16:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc0870640 1 -- 172.21.15.32:0/472672568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb80a7050 con 0x7fbfbc15cf00 2023-10-16T08:16:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc0870640 1 -- 172.21.15.32:0/472672568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80ab780 con 0x7fbfbc15cf00 2023-10-16T08:16:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/472672568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc15cf00 msgr2=0x7fbfbc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/472672568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc15cf00 0x7fbfbc0cd9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0b5490 tx=0x7fbfb801c1b0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/472672568 shutdown_connections 2023-10-16T08:16:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/472672568 >> v2:172.21.15.186:3300/0 conn(0x7fbfbc0cdee0 0x7fbfbc0d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/472672568 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc0d1240 0x7fbfbc06ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/472672568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc15cf00 0x7fbfbc0cd9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/472672568 >> 172.21.15.32:0/472672568 conn(0x7fbfbc0b4470 msgr2=0x7fbfbc0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/472672568 shutdown_connections 2023-10-16T08:16:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.475+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/472672568 wait complete. 2023-10-16T08:16:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 Processor -- start 2023-10-16T08:16:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- start start 2023-10-16T08:16:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 0x7fbfbc1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfbc0d1240 0x7fbfbc15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc15cf00 0x7fbfbc15b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc153910 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0ca400 con 0x7fbfbc0d1240 2023-10-16T08:16:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0aa190 con 0x7fbfbc15cf00 2023-10-16T08:16:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 0x7fbfbc1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 0x7fbfbc1533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41224/0 (socket says 172.21.15.32:41224) 2023-10-16T08:16:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 -- 172.21.15.32:0/3275305798 learned_addr learned my addr 172.21.15.32:0/3275305798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1071640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfbc0d1240 0x7fbfbc15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc2073640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc15cf00 0x7fbfbc15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 -- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc15cf00 msgr2=0x7fbfbc15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc15cf00 0x7fbfbc15b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 -- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfbc0d1240 msgr2=0x7fbfbc15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfbc0d1240 0x7fbfbc15aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 -- 172.21.15.32:0/3275305798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80094d0 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1071640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfbc0d1240 0x7fbfbc15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc2073640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc15cf00 0x7fbfbc15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc1872640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 0x7fbfbc1533d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbfb801a860 tx=0x7fbfb8018f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80abd50 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb80b6470 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc0c9ad0 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80b6800 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc0ca090 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.479+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc074010 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.483+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbfb80b2070 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.483+0000 7fbfb27fc640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbfa00664a0 0x7fbfa0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.483+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbfb811de00 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.483+0000 7fbfc1071640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbfa00664a0 0x7fbfa0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.483+0000 7fbfc1071640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbfa00664a0 0x7fbfa0068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc159d60 tx=0x7fbfac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.487+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbfb80eb650 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbfbc1593c0 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfb27fc640 1 -- 172.21.15.32:0/3275305798 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbfb80b0070 con 0x7fbfbc0cdee0 2023-10-16T08:16:19.648 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:19.648 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-10-16T08:16:19.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:16:19.649 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-10-16T08:16:19.649 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-10-16T08:16:19.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:16:19.649 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-10-16T08:16:19.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbfa00664a0 msgr2=0x7fbfa0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbfa00664a0 0x7fbfa0068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc159d60 tx=0x7fbfac008040 comp rx=0 tx=0).stop 2023-10-16T08:16:19.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 msgr2=0x7fbfbc1533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 0x7fbfbc1533d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbfb801a860 tx=0x7fbfb8018f50 comp rx=0 tx=0).stop 2023-10-16T08:16:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 shutdown_connections 2023-10-16T08:16:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbfa00664a0 0x7fbfa0068960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbfbc15cf00 0x7fbfbc15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfbc0d1240 0x7fbfbc15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 --2- 172.21.15.32:0/3275305798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfbc0cdee0 0x7fbfbc1533d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 >> 172.21.15.32:0/3275305798 conn(0x7fbfbc0b4470 msgr2=0x7fbfbc0b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 shutdown_connections 2023-10-16T08:16:19.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.643+0000 7fbfc3ad6640 1 -- 172.21.15.32:0/3275305798 wait complete. 2023-10-16T08:16:19.668 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T08:16:19.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T08:16:19.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efce5491640 1 Processor -- start 2023-10-16T08:16:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efce5491640 1 -- start start 2023-10-16T08:16:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efce5491640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efce00cdc80 0x7efce00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efce5491640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00ce460 con 0x7efce00d2570 2023-10-16T08:16:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efce5491640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce015da30 con 0x7efce015dec0 2023-10-16T08:16:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efce5491640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00cd830 con 0x7efce00cdc80 2023-10-16T08:16:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efcdeffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efce00d2570 0x7efce0161210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43306/0 (socket says 172.21.15.32:43306) 2023-10-16T08:16:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efcdeffd640 1 -- 172.21.15.32:0/1805294942 learned_addr learned my addr 172.21.15.32:0/1805294942 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efcd7fff640 1 --2- 172.21.15.32:0/1805294942 >> v2:172.21.15.186:3300/0 conn(0x7efce00cdc80 0x7efce00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124090680 0 0) 0x7efce00ce460 con 0x7efce00d2570 2023-10-16T08:16:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.811+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efccc003530 con 0x7efce00d2570 2023-10-16T08:16:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 844430136 0 0) 0x7efccc003530 con 0x7efce00d2570 2023-10-16T08:16:19.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 >> v2:172.21.15.186:3300/0 conn(0x7efce00cdc80 msgr2=0x7efce00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 --2- 172.21.15.32:0/1805294942 >> v2:172.21.15.186:3300/0 conn(0x7efce00cdc80 0x7efce00cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 >> v1:172.21.15.186:6789/0 conn(0x7efce015dec0 legacy=0x7efce00d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00ce180 con 0x7efce00d2570 2023-10-16T08:16:19.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7efcd0001680 con 0x7efce00d2570 2023-10-16T08:16:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 >> v1:172.21.15.32:6789/0 conn(0x7efce00d2570 legacy=0x7efce0161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc003980 0x7efccc003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efccc004380 0x7efccc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc005050 0x7efccc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00cd830 con 0x7efccc005050 2023-10-16T08:16:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00bfb0 con 0x7efccc004380 2023-10-16T08:16:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00c170 con 0x7efccc003980 2023-10-16T08:16:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcd7fff640 1 --2- 172.21.15.32:0/1805294942 >> v2:172.21.15.186:3300/0 conn(0x7efce00cdc80 0x7efce00cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc005050 0x7efccc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdeffd640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc003980 0x7efccc003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcd7fff640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efccc004380 0x7efccc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 -- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc003980 msgr2=0x7efccc003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc003980 0x7efccc003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 -- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efccc004380 msgr2=0x7efccc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efccc004380 0x7efccc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 -- 172.21.15.32:0/1805294942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc003530 con 0x7efccc005050 2023-10-16T08:16:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 -- 172.21.15.32:0/1805294942 reap_dead start 2023-10-16T08:16:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.815+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc005050 0x7efccc00b990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efcd8002010 tx=0x7efcd801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524c2aa843d9e939 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0007ea0 con 0x7efccc005050 2023-10-16T08:16:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd80a57b0 con 0x7efccc005050 2023-10-16T08:16:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcde7fc640 1 -- 172.21.15.32:0/1805294942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80a5b60 con 0x7efccc005050 2023-10-16T08:16:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- 172.21.15.32:0/1805294942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc005050 msgr2=0x7efccc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc005050 0x7efccc00b990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efcd8002010 tx=0x7efcd801c8c0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- 172.21.15.32:0/1805294942 shutdown_connections 2023-10-16T08:16:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 --2- 172.21.15.32:0/1805294942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc005050 0x7efccc00b990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- 172.21.15.32:0/1805294942 >> 172.21.15.32:0/1805294942 conn(0x7efce00715c0 msgr2=0x7efce00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- 172.21.15.32:0/1805294942 shutdown_connections 2023-10-16T08:16:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- 172.21.15.32:0/1805294942 wait complete. 2023-10-16T08:16:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 Processor -- start 2023-10-16T08:16:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- start start 2023-10-16T08:16:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc005050 0x7efce0157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efce01f22d0 0x7efce01f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 0x7efce00cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c9e90 con 0x7efce00caf30 2023-10-16T08:16:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00ca170 con 0x7efce01f22d0 2023-10-16T08:16:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efce5491640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00ca450 con 0x7efccc005050 2023-10-16T08:16:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 0x7efce00cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 0x7efce00cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41240/0 (socket says 172.21.15.32:41240) 2023-10-16T08:16:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcdf7fe640 1 -- 172.21.15.32:0/38089001 learned_addr learned my addr 172.21.15.32:0/38089001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcd7fff640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efce01f22d0 0x7efce01f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.819+0000 7efcdeffd640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc005050 0x7efce0157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcdf7fe640 1 -- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc005050 msgr2=0x7efce0157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc005050 0x7efce0157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcdf7fe640 1 -- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efce01f22d0 msgr2=0x7efce01f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efce01f22d0 0x7efce01f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcdf7fe640 1 -- 172.21.15.32:0/38089001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00ca730 con 0x7efce00caf30 2023-10-16T08:16:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcd7fff640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efce01f22d0 0x7efce01f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcdf7fe640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 0x7efce00cb3b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcd801c6f0 tx=0x7efccc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80a5b30 con 0x7efce00caf30 2023-10-16T08:16:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd80a5c90 con 0x7efce00caf30 2023-10-16T08:16:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00caa10 con 0x7efce00caf30 2023-10-16T08:16:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80050e0 con 0x7efce00caf30 2023-10-16T08:16:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01586e0 con 0x7efce00caf30 2023-10-16T08:16:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce004e170 con 0x7efce00caf30 2023-10-16T08:16:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efcd8005260 con 0x7efce00caf30 2023-10-16T08:16:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.823+0000 7efcd77fe640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efcc00664d0 0x7efcc0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.827+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efcd811d1a0 con 0x7efce00caf30 2023-10-16T08:16:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.827+0000 7efcdeffd640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efcc00664d0 0x7efcc0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.827+0000 7efcdeffd640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efcc00664d0 0x7efcc0068990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c10 tx=0x7efcd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.831+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efcd80ea8d0 con 0x7efce00caf30 2023-10-16T08:16:19.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:19.987+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efce01586e0 con 0x7efce00caf30 2023-10-16T08:16:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efcd77fe640 1 -- 172.21.15.32:0/38089001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7efce01586e0 con 0x7efce00caf30 2023-10-16T08:16:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-10-16T08:16:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efcc00664d0 msgr2=0x7efcc0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efcc00664d0 0x7efcc0068990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c10 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2023-10-16T08:16:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 msgr2=0x7efce00cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 0x7efce00cb3b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcd801c6f0 tx=0x7efccc005800 comp rx=0 tx=0).stop 2023-10-16T08:16:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 shutdown_connections 2023-10-16T08:16:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efce00caf30 0x7efce00cb3b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efce01f22d0 0x7efce01f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efcc00664d0 0x7efcc0068990 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c10 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2023-10-16T08:16:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 --2- 172.21.15.32:0/38089001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efccc005050 0x7efce0157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.915+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 >> 172.21.15.32:0/38089001 conn(0x7efce00715c0 msgr2=0x7efce0159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.919+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 shutdown_connections 2023-10-16T08:16:20.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:20.919+0000 7efce5491640 1 -- 172.21.15.32:0/38089001 wait complete. 2023-10-16T08:16:20.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-16T08:16:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55abc3f640 1 Processor -- start 2023-10-16T08:16:21.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55abc3f640 1 -- start start 2023-10-16T08:16:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55abc3f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 0x7f55a4167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55abc3f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a4161720 con 0x7f55a40b85d0 2023-10-16T08:16:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55abc3f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a4160ba0 con 0x7f55a40b89b0 2023-10-16T08:16:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55abc3f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4160e80 con 0x7f55a40d0300 2023-10-16T08:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55a40b85d0 0x7f55a415dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43312/0 (socket says 172.21.15.32:43312) 2023-10-16T08:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 -- 172.21.15.32:0/3254521680 learned_addr learned my addr 172.21.15.32:0/3254521680 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55aa1dc640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 0x7f55a4167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937208801 0 0) 0x7f55a4161720 con 0x7f55a40b85d0 2023-10-16T08:16:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5590003530 con 0x7f55a40b85d0 2023-10-16T08:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836861387 0 0) 0x7f55a4160ba0 con 0x7f55a40b89b0 2023-10-16T08:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a4161720 con 0x7f55a40b89b0 2023-10-16T08:16:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848181179 0 0) 0x7f5590003530 con 0x7f55a40b85d0 2023-10-16T08:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 msgr2=0x7f55a4167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 0x7f55a4167b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 >> v1:172.21.15.186:6789/0 conn(0x7f55a40b89b0 legacy=0x7f55a40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4161160 con 0x7f55a40b85d0 2023-10-16T08:16:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f55a0001680 con 0x7f55a40b85d0 2023-10-16T08:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 >> v1:172.21.15.32:6789/0 conn(0x7f55a40b85d0 legacy=0x7f55a415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f5590003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55900047d0 0x7f5590004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 0x7f5590005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4160e80 con 0x7f5590003b40 2023-10-16T08:16:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559000ba40 con 0x7f55900047d0 2023-10-16T08:16:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559000bc00 con 0x7f55a40d0300 2023-10-16T08:16:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f5590003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55aa1dc640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 0x7f5590005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a91da640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55900047d0 0x7f5590004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 -- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 msgr2=0x7f5590005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a40d0300 0x7f5590005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 -- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55900047d0 msgr2=0x7f5590004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55900047d0 0x7f5590004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a99db640 1 -- 172.21.15.32:0/3254521680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5590003530 con 0x7f5590003b40 2023-10-16T08:16:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.103+0000 7f55a91da640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55900047d0 0x7f5590004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55a99db640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f5590003f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55a0001ec0 tx=0x7f55a001d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24da781d729e7a21 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00058e0 con 0x7f5590003b40 2023-10-16T08:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a0005a60 con 0x7f5590003b40 2023-10-16T08:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55a89d9640 1 -- 172.21.15.32:0/3254521680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0005e50 con 0x7f5590003b40 2023-10-16T08:16:21.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55abc3f640 1 -- 172.21.15.32:0/3254521680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 msgr2=0x7f5590003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3254521680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f5590003f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55a0001ec0 tx=0x7f55a001d5b0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55aa1dc640 1 -- 172.21.15.32:0/3254521680 reap_dead start 2023-10-16T08:16:21.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55abc3f640 1 -- 172.21.15.32:0/3254521680 shutdown_connections 2023-10-16T08:16:21.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55abc3f640 1 -- 172.21.15.32:0/3254521680 >> 172.21.15.32:0/3254521680 conn(0x7f55a406b750 msgr2=0x7f55a406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:21.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55abc3f640 1 -- 172.21.15.32:0/3254521680 shutdown_connections 2023-10-16T08:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.107+0000 7f55abc3f640 1 -- 172.21.15.32:0/3254521680 wait complete. 2023-10-16T08:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 Processor -- start 2023-10-16T08:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 -- start start 2023-10-16T08:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f55a415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a414eb60 0x7f55a4153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a414e350 0x7f55a40b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4152ef0 con 0x7f5590003b40 2023-10-16T08:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a415ee30 con 0x7f55a414e350 2023-10-16T08:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4153ed0 con 0x7f55a414eb60 2023-10-16T08:16:21.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f55a415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f55a415e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41250/0 (socket says 172.21.15.32:41250) 2023-10-16T08:16:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 -- 172.21.15.32:0/3258668299 learned_addr learned my addr 172.21.15.32:0/3258668299 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55aa1dc640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a414e350 0x7f55a40b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a91da640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a414eb60 0x7f55a4153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 -- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a414eb60 msgr2=0x7f55a4153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a414eb60 0x7f55a4153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 -- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a414e350 msgr2=0x7f55a40b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a414e350 0x7f55a40b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 -- 172.21.15.32:0/3258668299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a415f0d0 con 0x7f5590003b40 2023-10-16T08:16:21.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55a99db640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f55a415e8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55a0007040 tx=0x7f5590005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0005e30 con 0x7f5590003b40 2023-10-16T08:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a00b2070 con 0x7f5590003b40 2023-10-16T08:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00047f0 con 0x7f5590003b40 2023-10-16T08:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a415f3b0 con 0x7f5590003b40 2023-10-16T08:16:21.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.111+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a4153470 con 0x7f5590003b40 2023-10-16T08:16:21.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.115+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a415f540 con 0x7f5590003b40 2023-10-16T08:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.115+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55a0004950 con 0x7f5590003b40 2023-10-16T08:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.115+0000 7f559a7fc640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55880664d0 0x7f5588068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.115+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f55a011d420 con 0x7f5590003b40 2023-10-16T08:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.115+0000 7f55a91da640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55880664d0 0x7f5588068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:21.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.115+0000 7f55a91da640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55880664d0 0x7f5588068990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55a4151cd0 tx=0x7f5594009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:21.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.119+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f55a00ea9e0 con 0x7f5590003b40 2023-10-16T08:16:21.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.279+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f55a404e170 con 0x7f5590003b40 2023-10-16T08:16:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.915+0000 7f559a7fc640 1 -- 172.21.15.32:0/3258668299 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f55a00ee000 con 0x7f5590003b40 2023-10-16T08:16:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-10-16T08:16:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55880664d0 msgr2=0x7f5588068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55880664d0 0x7f5588068990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55a4151cd0 tx=0x7f5594009040 comp rx=0 tx=0).stop 2023-10-16T08:16:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 msgr2=0x7f55a415e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f55a415e8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55a0007040 tx=0x7f5590005800 comp rx=0 tx=0).stop 2023-10-16T08:16:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 shutdown_connections 2023-10-16T08:16:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a414e350 0x7f55a40b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55880664d0 0x7f5588068990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55a414eb60 0x7f55a4153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 --2- 172.21.15.32:0/3258668299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5590003b40 0x7f55a415e8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:21.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.919+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 >> 172.21.15.32:0/3258668299 conn(0x7f55a406b750 msgr2=0x7f55a4155820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:21.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.923+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 shutdown_connections 2023-10-16T08:16:21.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:21.923+0000 7f55abc3f640 1 -- 172.21.15.32:0/3258668299 wait complete. 2023-10-16T08:16:21.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T08:16:22.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.111+0000 7f6816e6b640 1 Processor -- start 2023-10-16T08:16:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.111+0000 7f6816e6b640 1 -- start start 2023-10-16T08:16:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f6816e6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f681015cba0 0x7f68100d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f6816e6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68100b84e0 0x7f68100cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f6816e6b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68100d1670 con 0x7f68100d0040 2023-10-16T08:16:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f6816e6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100d1460 con 0x7f68100b84e0 2023-10-16T08:16:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f6816e6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100d04b0 con 0x7f681015cba0 2023-10-16T08:16:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68100b84e0 0x7f68100cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68100b84e0 0x7f68100cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41266/0 (socket says 172.21.15.32:41266) 2023-10-16T08:16:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 -- 172.21.15.32:0/2666467633 learned_addr learned my addr 172.21.15.32:0/2666467633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f6814c07640 1 --2- 172.21.15.32:0/2666467633 >> v2:172.21.15.186:3300/0 conn(0x7f681015cba0 0x7f68100d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 -- 172.21.15.32:0/2666467633 >> v2:172.21.15.186:3300/0 conn(0x7f681015cba0 msgr2=0x7f68100d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 --2- 172.21.15.32:0/2666467633 >> v2:172.21.15.186:3300/0 conn(0x7f681015cba0 0x7f68100d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 -- 172.21.15.32:0/2666467633 >> v1:172.21.15.186:6789/0 conn(0x7f68100d0040 legacy=0x7f68100d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 -- 172.21.15.32:0/2666467633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100d1950 con 0x7f68100b84e0 2023-10-16T08:16:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680ffff640 1 --2- 172.21.15.32:0/2666467633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100b84e0 0x7f68100cfb00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68100b4890 tx=0x7f67fc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baf0b07d33f08d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680f7fe640 1 -- 172.21.15.32:0/2666467633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc004230 con 0x7f68100b84e0 2023-10-16T08:16:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680f7fe640 1 -- 172.21.15.32:0/2666467633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc0a7050 con 0x7f68100b84e0 2023-10-16T08:16:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.115+0000 7f680f7fe640 1 -- 172.21.15.32:0/2666467633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc003620 con 0x7f68100b84e0 2023-10-16T08:16:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- 172.21.15.32:0/2666467633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100b84e0 msgr2=0x7f68100cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 --2- 172.21.15.32:0/2666467633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100b84e0 0x7f68100cfb00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68100b4890 tx=0x7f67fc01c430 comp rx=0 tx=0).stop 2023-10-16T08:16:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- 172.21.15.32:0/2666467633 shutdown_connections 2023-10-16T08:16:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 --2- 172.21.15.32:0/2666467633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100b84e0 0x7f68100cfb00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 --2- 172.21.15.32:0/2666467633 >> v2:172.21.15.186:3300/0 conn(0x7f681015cba0 0x7f68100d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- 172.21.15.32:0/2666467633 >> 172.21.15.32:0/2666467633 conn(0x7f68100719c0 msgr2=0x7f6810071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- 172.21.15.32:0/2666467633 shutdown_connections 2023-10-16T08:16:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- 172.21.15.32:0/2666467633 wait complete. 2023-10-16T08:16:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 Processor -- start 2023-10-16T08:16:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- start start 2023-10-16T08:16:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68100b84e0 0x7f681014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68100d0040 0x7f68101565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 0x7f68101545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68101503b0 con 0x7f681015cba0 2023-10-16T08:16:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681015d990 con 0x7f68100b84e0 2023-10-16T08:16:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6816e6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681015dc70 con 0x7f68100d0040 2023-10-16T08:16:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6815408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 0x7f68101545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6815408640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 0x7f68101545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41274/0 (socket says 172.21.15.32:41274) 2023-10-16T08:16:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6815408640 1 -- 172.21.15.32:0/1334688535 learned_addr learned my addr 172.21.15.32:0/1334688535 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f6814c07640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68100b84e0 0x7f681014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.119+0000 7f680ffff640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68100d0040 0x7f68101565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:22.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6815408640 1 -- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68100d0040 msgr2=0x7f68101565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:22.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6815408640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68100d0040 0x7f68101565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6815408640 1 -- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68100b84e0 msgr2=0x7f681014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:22.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6815408640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68100b84e0 0x7f681014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6815408640 1 -- 172.21.15.32:0/1334688535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0094d0 con 0x7f681015cba0 2023-10-16T08:16:22.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6814c07640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68100b84e0 0x7f681014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:22.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6815408640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 0x7f68101545a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f680800c780 tx=0x7f680800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:22.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808004b70 con 0x7f681015cba0 2023-10-16T08:16:22.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680800ce90 con 0x7f681015cba0 2023-10-16T08:16:22.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800a850 con 0x7f681015cba0 2023-10-16T08:16:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681015df50 con 0x7f681015cba0 2023-10-16T08:16:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681015e510 con 0x7f681015cba0 2023-10-16T08:16:22.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681004e170 con 0x7f681015cba0 2023-10-16T08:16:22.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f680800a9b0 con 0x7f681015cba0 2023-10-16T08:16:22.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f680d7fa640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67f40664a0 0x7f67f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:22.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.123+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6808083ff0 con 0x7f681015cba0 2023-10-16T08:16:22.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.127+0000 7f6814c07640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67f40664a0 0x7f67f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:22.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.127+0000 7f6814c07640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67f40664a0 0x7f67f4068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6810160280 tx=0x7f6804008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:22.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.131+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68080b5090 con 0x7f681015cba0 2023-10-16T08:16:22.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.319+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6810071da0 con 0x7f681015cba0 2023-10-16T08:16:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.951+0000 7f680d7fa640 1 -- 172.21.15.32:0/1334688535 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6808051450 con 0x7f681015cba0 2023-10-16T08:16:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2023-10-16T08:16:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.971+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67f40664a0 msgr2=0x7f67f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.971+0000 7f6816e6b640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67f40664a0 0x7f67f4068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6810160280 tx=0x7f6804008040 comp rx=0 tx=0).stop 2023-10-16T08:16:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.971+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 msgr2=0x7f68101545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.971+0000 7f6816e6b640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 0x7f68101545a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f680800c780 tx=0x7f680800cc40 comp rx=0 tx=0).stop 2023-10-16T08:16:22.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 shutdown_connections 2023-10-16T08:16:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f681015cba0 0x7f68101545a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67f40664a0 0x7f67f4068960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68100d0040 0x7f68101565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 --2- 172.21.15.32:0/1334688535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68100b84e0 0x7f681014f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:22.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 >> 172.21.15.32:0/1334688535 conn(0x7f68100719c0 msgr2=0x7f6810159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 shutdown_connections 2023-10-16T08:16:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:22.975+0000 7f6816e6b640 1 -- 172.21.15.32:0/1334688535 wait complete. 2023-10-16T08:16:23.014 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-16T08:16:23.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-16T08:16:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.159+0000 7f545dc6b640 1 Processor -- start 2023-10-16T08:16:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.159+0000 7f545dc6b640 1 -- start start 2023-10-16T08:16:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f545dc6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54581441a0 0x7f54580b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f545dc6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f54580b1a90 0x7f54580cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f545dc6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580cb360 0x7f545806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f545dc6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580b1f20 con 0x7f54581441a0 2023-10-16T08:16:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f545dc6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545815e5f0 con 0x7f54580b1a90 2023-10-16T08:16:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f545dc6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545815e8d0 con 0x7f54580cb360 2023-10-16T08:16:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54581441a0 0x7f54580b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54581441a0 0x7f54580b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50468/0 (socket says 172.21.15.32:50468) 2023-10-16T08:16:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 -- 172.21.15.32:0/3409324227 learned_addr learned my addr 172.21.15.32:0/3409324227 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f5456ffd640 1 --2- 172.21.15.32:0/3409324227 >> v2:172.21.15.186:3300/0 conn(0x7f54580b1a90 0x7f54580cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f5457fff640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580cb360 0x7f545806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 -- 172.21.15.32:0/3409324227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580cb360 msgr2=0x7f545806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580cb360 0x7f545806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 -- 172.21.15.32:0/3409324227 >> v2:172.21.15.186:3300/0 conn(0x7f54580b1a90 msgr2=0x7f54580cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 --2- 172.21.15.32:0/3409324227 >> v2:172.21.15.186:3300/0 conn(0x7f54580b1a90 0x7f54580cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f54577fe640 1 -- 172.21.15.32:0/3409324227 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545815ebb0 con 0x7f54581441a0 2023-10-16T08:16:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f5456ffd640 1 --2- 172.21.15.32:0/3409324227 >> v2:172.21.15.186:3300/0 conn(0x7f54580b1a90 0x7f54580cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.163+0000 7f5457fff640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580cb360 0x7f545806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f54577fe640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54581441a0 0x7f54580b1550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f545806d580 tx=0x7f544801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a597f5c3a0b2cc0f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f54567fc640 1 -- 172.21.15.32:0/3409324227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54480ab600 con 0x7f54581441a0 2023-10-16T08:16:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f54567fc640 1 -- 172.21.15.32:0/3409324227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54480a7050 con 0x7f54581441a0 2023-10-16T08:16:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f54567fc640 1 -- 172.21.15.32:0/3409324227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54480ab980 con 0x7f54581441a0 2023-10-16T08:16:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- 172.21.15.32:0/3409324227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54581441a0 msgr2=0x7f54580b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54581441a0 0x7f54580b1550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f545806d580 tx=0x7f544801c240 comp rx=0 tx=0).stop 2023-10-16T08:16:23.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- 172.21.15.32:0/3409324227 shutdown_connections 2023-10-16T08:16:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3409324227 >> v2:172.21.15.186:3300/0 conn(0x7f54580b1a90 0x7f54580cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580cb360 0x7f545806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3409324227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54581441a0 0x7f54580b1550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- 172.21.15.32:0/3409324227 >> 172.21.15.32:0/3409324227 conn(0x7f545806a820 msgr2=0x7f54580b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- 172.21.15.32:0/3409324227 shutdown_connections 2023-10-16T08:16:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- 172.21.15.32:0/3409324227 wait complete. 2023-10-16T08:16:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 Processor -- start 2023-10-16T08:16:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- start start 2023-10-16T08:16:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580b1a90 0x7f5458138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 0x7f5458140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54581441a0 0x7f5458139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5458138a20 con 0x7f54580cb360 2023-10-16T08:16:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5458140e60 con 0x7f54581441a0 2023-10-16T08:16:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.167+0000 7f545dc6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5458138c90 con 0x7f54580b1a90 2023-10-16T08:16:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 0x7f5458140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 0x7f5458140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50482/0 (socket says 172.21.15.32:50482) 2023-10-16T08:16:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 -- 172.21.15.32:0/3694596042 learned_addr learned my addr 172.21.15.32:0/3694596042 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5457fff640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54581441a0 0x7f5458139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f54577fe640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580b1a90 0x7f5458138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 -- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580b1a90 msgr2=0x7f5458138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580b1a90 0x7f5458138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 -- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54581441a0 msgr2=0x7f5458139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54581441a0 0x7f5458139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 -- 172.21.15.32:0/3694596042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480094d0 con 0x7f54580cb360 2023-10-16T08:16:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f5456ffd640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 0x7f5458140560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e810 tx=0x7f544c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c004030 con 0x7f54580cb360 2023-10-16T08:16:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c004190 con 0x7f54580cb360 2023-10-16T08:16:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c004520 con 0x7f54580cb360 2023-10-16T08:16:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5458138f70 con 0x7f54580cb360 2023-10-16T08:16:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5458141140 con 0x7f54580cb360 2023-10-16T08:16:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54580c8f40 con 0x7f54580cb360 2023-10-16T08:16:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.171+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f544c00c6d0 con 0x7f54580cb360 2023-10-16T08:16:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.175+0000 7f543ffff640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54440664d0 0x7f5444068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.175+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f544c0839f0 con 0x7f54580cb360 2023-10-16T08:16:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.175+0000 7f54577fe640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54440664d0 0x7f5444068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.175+0000 7f54577fe640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54440664d0 0x7f5444068990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54480ad000 tx=0x7f5448008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.179+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f544c050e80 con 0x7f54580cb360 2023-10-16T08:16:23.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.343+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f545804e170 con 0x7f54580cb360 2023-10-16T08:16:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.935+0000 7f543ffff640 1 -- 172.21.15.32:0/3694596042 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f544c0544a0 con 0x7f54580cb360 2023-10-16T08:16:23.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54440664d0 msgr2=0x7f5444068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:23.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54440664d0 0x7f5444068990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54480ad000 tx=0x7f5448008040 comp rx=0 tx=0).stop 2023-10-16T08:16:23.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 msgr2=0x7f5458140560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:23.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 0x7f5458140560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e810 tx=0x7f544c00ecd0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 shutdown_connections 2023-10-16T08:16:23.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54581441a0 0x7f5458139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54580cb360 0x7f5458140560 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e810 tx=0x7f544c00ecd0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54440664d0 0x7f5444068990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 --2- 172.21.15.32:0/3694596042 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54580b1a90 0x7f5458138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:23.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 >> 172.21.15.32:0/3694596042 conn(0x7f545806a820 msgr2=0x7f5458157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:23.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 shutdown_connections 2023-10-16T08:16:23.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:23.939+0000 7f545dc6b640 1 -- 172.21.15.32:0/3694596042 wait complete. 2023-10-16T08:16:23.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T08:16:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 Processor -- start 2023-10-16T08:16:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 -- start start 2023-10-16T08:16:24.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7fec15cc10 0x7f7fec0b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:24.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec0b5ba0 0x7f7fec0d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec15cff0 con 0x7f7fec0cd800 2023-10-16T08:16:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec0b8b80 con 0x7f7fec15cc10 2023-10-16T08:16:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff35c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec0b63f0 con 0x7f7fec0b5ba0 2023-10-16T08:16:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff1b61640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7fec0cd800 0x7f7fec0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40850/0 (socket says 172.21.15.32:40850) 2023-10-16T08:16:24.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.119+0000 7f7ff1b61640 1 -- 172.21.15.32:0/3783702555 learned_addr learned my addr 172.21.15.32:0/3783702555 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3783702555 >> v2:172.21.15.186:3300/0 conn(0x7f7fec15cc10 0x7f7fec0b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:24.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff0b5f640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec0b5ba0 0x7f7fec0d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7fe3fff640 1 -- 172.21.15.32:0/3783702555 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208905423 0 0) 0x7f7fec15cff0 con 0x7f7fec0cd800 2023-10-16T08:16:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7fe3fff640 1 -- 172.21.15.32:0/3783702555 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd8003530 con 0x7f7fec0cd800 2023-10-16T08:16:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff1360640 1 -- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec0b5ba0 msgr2=0x7f7fec0d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec0b5ba0 0x7f7fec0d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff1360640 1 -- 172.21.15.32:0/3783702555 >> v1:172.21.15.32:6789/0 conn(0x7f7fec0cd800 legacy=0x7f7fec0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff1360640 1 -- 172.21.15.32:0/3783702555 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0b66d0 con 0x7f7fec15cc10 2023-10-16T08:16:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff0b5f640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec0b5ba0 0x7f7fec0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec15cc10 0x7f7fec0b8640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80097c0 tx=0x7f7fe801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f04e758f9d8917 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7fe3fff640 1 -- 172.21.15.32:0/3783702555 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80a8970 con 0x7f7fec15cc10 2023-10-16T08:16:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7fe3fff640 1 -- 172.21.15.32:0/3783702555 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80a8ad0 con 0x7f7fec15cc10 2023-10-16T08:16:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.123+0000 7f7fe3fff640 1 -- 172.21.15.32:0/3783702555 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80a8e60 con 0x7f7fec15cc10 2023-10-16T08:16:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec15cc10 msgr2=0x7f7fec0b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec15cc10 0x7f7fec0b8640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80097c0 tx=0x7f7fe801ec70 comp rx=0 tx=0).stop 2023-10-16T08:16:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3783702555 shutdown_connections 2023-10-16T08:16:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec0b5ba0 0x7f7fec0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3783702555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec15cc10 0x7f7fec0b8640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3783702555 >> 172.21.15.32:0/3783702555 conn(0x7f7fec0719c0 msgr2=0x7f7fec071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3783702555 shutdown_connections 2023-10-16T08:16:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3783702555 wait complete. 2023-10-16T08:16:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 Processor -- start 2023-10-16T08:16:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- start start 2023-10-16T08:16:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec0b5ba0 0x7f7fec157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 0x7f7fec0b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec15cc10 0x7f7fec0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec0ca920 con 0x7f7fec0cd800 2023-10-16T08:16:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec157d40 con 0x7f7fec0b5ba0 2023-10-16T08:16:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff35c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec15edd0 con 0x7f7fec15cc10 2023-10-16T08:16:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 0x7f7fec0b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 0x7f7fec0b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50490/0 (socket says 172.21.15.32:50490) 2023-10-16T08:16:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 -- 172.21.15.32:0/3923164011 learned_addr learned my addr 172.21.15.32:0/3923164011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec0b5ba0 0x7f7fec157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff1b61640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec15cc10 0x7f7fec0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 -- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec15cc10 msgr2=0x7f7fec0b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec15cc10 0x7f7fec0b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 -- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec0b5ba0 msgr2=0x7f7fec157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec0b5ba0 0x7f7fec157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff0b5f640 1 -- 172.21.15.32:0/3923164011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc009420 con 0x7f7fec0cd800 2023-10-16T08:16:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.127+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec0b5ba0 0x7f7fec157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7ff0b5f640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 0x7f7fec0b7330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00ebe0 tx=0x7f7fdc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00eee0 con 0x7f7fec0cd800 2023-10-16T08:16:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc00cc30 con 0x7f7fec0cd800 2023-10-16T08:16:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc013470 con 0x7f7fec0cd800 2023-10-16T08:16:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8009420 con 0x7f7fec0cd800 2023-10-16T08:16:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec15f390 con 0x7f7fec0cd800 2023-10-16T08:16:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec04e170 con 0x7f7fec0cd800 2023-10-16T08:16:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7fdc0135d0 con 0x7f7fec0cd800 2023-10-16T08:16:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7fe1ffb640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7fd00664a0 0x7f7fd0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f7fdc084310 con 0x7f7fec0cd800 2023-10-16T08:16:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.131+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7fd00664a0 0x7f7fd0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.135+0000 7f7ff1360640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7fd00664a0 0x7f7fd0068960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80b2000 tx=0x7f7fe8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.135+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7fdc051610 con 0x7f7fec0cd800 2023-10-16T08:16:24.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:24.295+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7fec0d22c0 con 0x7f7fec0cd800 2023-10-16T08:16:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.971+0000 7f7fe1ffb640 1 -- 172.21.15.32:0/3923164011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f7fdc054c30 con 0x7f7fec0cd800 2023-10-16T08:16:25.976 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data_ec' created 2023-10-16T08:16:25.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.975+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7fd00664a0 msgr2=0x7f7fd0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:25.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.975+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7fd00664a0 0x7f7fd0068960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80b2000 tx=0x7f7fe8008040 comp rx=0 tx=0).stop 2023-10-16T08:16:25.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.975+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 msgr2=0x7f7fec0b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:25.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.975+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 0x7f7fec0b7330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00ebe0 tx=0x7f7fdc00c520 comp rx=0 tx=0).stop 2023-10-16T08:16:25.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 shutdown_connections 2023-10-16T08:16:25.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7fd00664a0 0x7f7fd0068960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:25.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7fec15cc10 0x7f7fec0b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:25.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fec0cd800 0x7f7fec0b7330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:25.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 --2- 172.21.15.32:0/3923164011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec0b5ba0 0x7f7fec157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:25.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 >> 172.21.15.32:0/3923164011 conn(0x7f7fec0719c0 msgr2=0x7f7fec0cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:25.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 shutdown_connections 2023-10-16T08:16:25.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:25.979+0000 7f7ff35c4640 1 -- 172.21.15.32:0/3923164011 wait complete. 2023-10-16T08:16:26.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-16T08:16:26.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.159+0000 7f9b03faa640 1 Processor -- start 2023-10-16T08:16:26.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.159+0000 7f9b03faa640 1 -- start start 2023-10-16T08:16:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b03faa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 0x7f9afc161210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:26.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b03faa640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9afc0cdc80 0x7f9afc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b03faa640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0ce460 con 0x7f9afc15dec0 2023-10-16T08:16:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b03faa640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0cd830 con 0x7f9afc0cdc80 2023-10-16T08:16:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b03faa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc15da30 con 0x7f9afc0d2570 2023-10-16T08:16:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01d46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 0x7f9afc161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:26.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01545640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9afc0cdc80 0x7f9afc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01d46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 0x7f9afc161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:42188/0 (socket says 172.21.15.32:42188) 2023-10-16T08:16:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01d46640 1 -- 172.21.15.32:0/2792283892 learned_addr learned my addr 172.21.15.32:0/2792283892 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01545640 1 -- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 msgr2=0x7f9afc161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01545640 1 --2- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 0x7f9afc161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01545640 1 -- 172.21.15.32:0/2792283892 >> v1:172.21.15.186:6789/0 conn(0x7f9afc15dec0 legacy=0x7f9afc0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01545640 1 -- 172.21.15.32:0/2792283892 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc161750 con 0x7f9afc0cdc80 2023-10-16T08:16:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.163+0000 7f9b01d46640 1 --2- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 0x7f9afc161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b01545640 1 --2- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0cdc80 0x7f9afc0cf610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0097c0 tx=0x7f9aec01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c250609f4cd3830 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b00d44640 1 -- 172.21.15.32:0/2792283892 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0ad660 con 0x7f9afc0cdc80 2023-10-16T08:16:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b00d44640 1 -- 172.21.15.32:0/2792283892 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec0ad7c0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b00d44640 1 -- 172.21.15.32:0/2792283892 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0adb50 con 0x7f9afc0cdc80 2023-10-16T08:16:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 -- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0cdc80 msgr2=0x7f9afc0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 --2- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0cdc80 0x7f9afc0cf610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0097c0 tx=0x7f9aec01c450 comp rx=0 tx=0).stop 2023-10-16T08:16:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 -- 172.21.15.32:0/2792283892 shutdown_connections 2023-10-16T08:16:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 --2- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0cdc80 0x7f9afc0cf610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 --2- 172.21.15.32:0/2792283892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc0d2570 0x7f9afc161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 -- 172.21.15.32:0/2792283892 >> 172.21.15.32:0/2792283892 conn(0x7f9afc0715c0 msgr2=0x7f9afc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 -- 172.21.15.32:0/2792283892 shutdown_connections 2023-10-16T08:16:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 -- 172.21.15.32:0/2792283892 wait complete. 2023-10-16T08:16:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 Processor -- start 2023-10-16T08:16:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.167+0000 7f9b03faa640 1 -- start start 2023-10-16T08:16:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 0x7f9afc161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0d2570 0x7f9afc0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc15dec0 0x7f9afc1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc153cf0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc153fd0 con 0x7f9afc0d2570 2023-10-16T08:16:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1542b0 con 0x7f9afc15dec0 2023-10-16T08:16:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 0x7f9afc161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 0x7f9afc161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50498/0 (socket says 172.21.15.32:50498) 2023-10-16T08:16:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 -- 172.21.15.32:0/753552589 learned_addr learned my addr 172.21.15.32:0/753552589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01545640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0d2570 0x7f9afc0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b02547640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc15dec0 0x7f9afc1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 -- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc15dec0 msgr2=0x7f9afc1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc15dec0 0x7f9afc1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 -- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0d2570 msgr2=0x7f9afc0c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0d2570 0x7f9afc0c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 -- 172.21.15.32:0/753552589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80094d0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b01d46640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 0x7f9afc161bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e930 tx=0x7f9af800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800ca00 con 0x7f9afc0cdc80 2023-10-16T08:16:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af800cb60 con 0x7f9afc0cdc80 2023-10-16T08:16:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8013470 con 0x7f9afc0cdc80 2023-10-16T08:16:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec009420 con 0x7f9afc0cdc80 2023-10-16T08:16:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc154870 con 0x7f9afc0cdc80 2023-10-16T08:16:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc04e170 con 0x7f9afc0cdc80 2023-10-16T08:16:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.171+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9af80135d0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.175+0000 7f9af27fc640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ae00664a0 0x7f9ae0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.175+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f9af80844c0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.175+0000 7f9b01545640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ae00664a0 0x7f9ae0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.175+0000 7f9b01545640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ae00664a0 0x7f9ae0068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0af000 tx=0x7f9aec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.179+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9af80515d0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.335+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9afc1679a0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.340 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-10-16T08:16:26.335+0000 7f18d54bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:16:26.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.979+0000 7f9af27fc640 1 -- 172.21.15.32:0/753552589 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9af8054bf0 con 0x7f9afc0cdc80 2023-10-16T08:16:26.983 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 4 allow_ec_overwrites to true 2023-10-16T08:16:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ae00664a0 msgr2=0x7f9ae0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ae00664a0 0x7f9ae0068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0af000 tx=0x7f9aec0a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 msgr2=0x7f9afc161bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 0x7f9afc161bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e930 tx=0x7f9af800edf0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 shutdown_connections 2023-10-16T08:16:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ae00664a0 0x7f9ae0068960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9afc15dec0 0x7f9afc1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9afc0d2570 0x7f9afc0c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 --2- 172.21.15.32:0/753552589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9afc0cdc80 0x7f9afc161bb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.983+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 >> 172.21.15.32:0/753552589 conn(0x7f9afc0715c0 msgr2=0x7f9afc15a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.987+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 shutdown_connections 2023-10-16T08:16:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:26.987+0000 7f9b03faa640 1 -- 172.21.15.32:0/753552589 wait complete. 2023-10-16T08:16:27.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-16T08:16:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.175+0000 7f6bb9fb8640 1 Processor -- start 2023-10-16T08:16:27.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.175+0000 7f6bb9fb8640 1 -- start start 2023-10-16T08:16:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb9fb8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6bb40cd2c0 0x7f6bb40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb9fb8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40ceb10 con 0x7f6bb415ac20 2023-10-16T08:16:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb9fb8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb415b000 con 0x7f6bb40b3800 2023-10-16T08:16:27.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb9fb8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb40cdcb0 con 0x7f6bb40cd2c0 2023-10-16T08:16:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6bb415ac20 0x7f6bb40cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40866/0 (socket says 172.21.15.32:40866) 2023-10-16T08:16:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb37fe640 1 -- 172.21.15.32:0/2563256909 learned_addr learned my addr 172.21.15.32:0/2563256909 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb2ffd640 1 --2- 172.21.15.32:0/2563256909 >> v2:172.21.15.186:3300/0 conn(0x7f6bb40cd2c0 0x7f6bb40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722780262 0 0) 0x7f6bb40ceb10 con 0x7f6bb415ac20 2023-10-16T08:16:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba0003530 con 0x7f6bb415ac20 2023-10-16T08:16:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248205861 0 0) 0x7f6ba0003530 con 0x7f6bb415ac20 2023-10-16T08:16:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 >> v2:172.21.15.186:3300/0 conn(0x7f6bb40cd2c0 msgr2=0x7f6bb40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 --2- 172.21.15.32:0/2563256909 >> v2:172.21.15.186:3300/0 conn(0x7f6bb40cd2c0 0x7f6bb40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 >> v1:172.21.15.186:6789/0 conn(0x7f6bb40b3800 legacy=0x7f6bb40b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40cbd50 con 0x7f6bb415ac20 2023-10-16T08:16:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f6b9c001680 con 0x7f6bb415ac20 2023-10-16T08:16:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 >> v1:172.21.15.32:6789/0 conn(0x7f6bb415ac20 legacy=0x7f6bb40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba0003980 0x7f6ba0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00043a0 0x7f6ba0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba0005070 0x7f6ba000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40cdcb0 con 0x7f6ba00043a0 2023-10-16T08:16:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba000bfb0 con 0x7f6ba0005070 2023-10-16T08:16:27.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.179+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba000c170 con 0x7f6ba0003980 2023-10-16T08:16:27.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb37fe640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba0003980 0x7f6ba0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb2ffd640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00043a0 0x7f6ba0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb3fff640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba0005070 0x7f6ba000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb2ffd640 1 -- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba0003980 msgr2=0x7f6ba0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb2ffd640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba0003980 0x7f6ba0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb2ffd640 1 -- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba0005070 msgr2=0x7f6ba000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb2ffd640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba0005070 0x7f6ba000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb2ffd640 1 -- 172.21.15.32:0/2563256909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40cbd50 con 0x7f6ba00043a0 2023-10-16T08:16:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.183+0000 7f6bb3fff640 1 -- 172.21.15.32:0/2563256909 reap_dead start 2023-10-16T08:16:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb2ffd640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00043a0 0x7f6ba0004870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009ac0 tx=0x7f6ba401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42ace6bfc04bbcf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40a5620 con 0x7f6ba00043a0 2023-10-16T08:16:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40a5780 con 0x7f6ba00043a0 2023-10-16T08:16:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb27fc640 1 -- 172.21.15.32:0/2563256909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40a5b10 con 0x7f6ba00043a0 2023-10-16T08:16:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/2563256909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00043a0 msgr2=0x7f6ba0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00043a0 0x7f6ba0004870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009ac0 tx=0x7f6ba401ca30 comp rx=0 tx=0).stop 2023-10-16T08:16:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/2563256909 shutdown_connections 2023-10-16T08:16:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/2563256909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ba00043a0 0x7f6ba0004870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/2563256909 >> 172.21.15.32:0/2563256909 conn(0x7f6bb406e570 msgr2=0x7f6bb406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/2563256909 shutdown_connections 2023-10-16T08:16:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/2563256909 wait complete. 2023-10-16T08:16:27.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.187+0000 7f6bb9fb8640 1 Processor -- start 2023-10-16T08:16:27.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- start start 2023-10-16T08:16:27.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba00043a0 0x7f6bb40b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bb40c26c0 0x7f6bb40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 0x7f6bb40c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40c3900 con 0x7f6bb40c43c0 2023-10-16T08:16:27.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40c3080 con 0x7f6bb40c26c0 2023-10-16T08:16:27.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40c3340 con 0x7f6ba00043a0 2023-10-16T08:16:27.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 0x7f6bb40c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb37fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba00043a0 0x7f6bb40b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb2ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bb40c26c0 0x7f6bb40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 0x7f6bb40c4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50504/0 (socket says 172.21.15.32:50504) 2023-10-16T08:16:27.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 -- 172.21.15.32:0/1130276693 learned_addr learned my addr 172.21.15.32:0/1130276693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:27.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 -- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba00043a0 msgr2=0x7f6bb40b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:27.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba00043a0 0x7f6bb40b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 -- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bb40c26c0 msgr2=0x7f6bb40c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:27.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bb40c26c0 0x7f6bb40c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 -- 172.21.15.32:0/1130276693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40c3620 con 0x7f6bb40c43c0 2023-10-16T08:16:27.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb2ffd640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bb40c26c0 0x7f6bb40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:27.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb3fff640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 0x7f6bb40c4870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40725f0 tx=0x7f6bac00e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:27.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac006d50 con 0x7f6bb40c43c0 2023-10-16T08:16:27.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac003980 con 0x7f6bb40c43c0 2023-10-16T08:16:27.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac003d30 con 0x7f6bb40c43c0 2023-10-16T08:16:27.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40c8ca0 con 0x7f6bb40c43c0 2023-10-16T08:16:27.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40b45a0 con 0x7f6bb40c43c0 2023-10-16T08:16:27.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.191+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb40c8b00 con 0x7f6bb40c43c0 2023-10-16T08:16:27.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.195+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6bac005070 con 0x7f6bb40c43c0 2023-10-16T08:16:27.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.195+0000 7f6babfff640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6b940664d0 0x7f6b94068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:27.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.195+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f6bac083550 con 0x7f6bb40c43c0 2023-10-16T08:16:27.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.195+0000 7f6bb37fe640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6b940664d0 0x7f6b94068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:27.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.195+0000 7f6bb37fe640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6b940664d0 0x7f6b94068990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0017b0 tx=0x7f6b9c001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:27.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.199+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6bb40c8b00 con 0x7f6bb40c43c0 2023-10-16T08:16:27.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.391+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f6bb40c8b00 con 0x7f6bb40c43c0 2023-10-16T08:16:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.979+0000 7f6babfff640 1 -- 172.21.15.32:0/1130276693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6bb40c8b00 con 0x7f6bb40c43c0 2023-10-16T08:16:27.985 INFO:teuthology.orchestra.run.smithi032.stderr:added data pool 4 to fsmap 2023-10-16T08:16:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6b940664d0 msgr2=0x7f6b94068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6b940664d0 0x7f6b94068990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0017b0 tx=0x7f6b9c001380 comp rx=0 tx=0).stop 2023-10-16T08:16:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 msgr2=0x7f6bb40c4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 0x7f6bb40c4870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40725f0 tx=0x7f6bac00e920 comp rx=0 tx=0).stop 2023-10-16T08:16:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 shutdown_connections 2023-10-16T08:16:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb40c43c0 0x7f6bb40c4870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bb40c26c0 0x7f6bb40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6b940664d0 0x7f6b94068990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 --2- 172.21.15.32:0/1130276693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6ba00043a0 0x7f6bb40b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:27.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 >> 172.21.15.32:0/1130276693 conn(0x7f6bb406e570 msgr2=0x7f6bb406b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:27.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 shutdown_connections 2023-10-16T08:16:27.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:27.983+0000 7f6bb9fb8640 1 -- 172.21.15.32:0/1130276693 wait complete. 2023-10-16T08:16:28.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:28.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 Processor -- start 2023-10-16T08:16:28.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 -- start start 2023-10-16T08:16:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23a815ca70 0x7f23a80d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80d3550 0x7f23a80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a80c65f0 con 0x7f23a80cf5d0 2023-10-16T08:16:28.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a80ce820 con 0x7f23a815ca70 2023-10-16T08:16:28.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23ac89e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80cfa40 con 0x7f23a80d3550 2023-10-16T08:16:28.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23a659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23a815ca70 0x7f23a80d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23a6d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23a80cf5d0 0x7f23a80d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40874/0 (socket says 172.21.15.32:40874) 2023-10-16T08:16:28.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.155+0000 7f23a6d9d640 1 -- 172.21.15.32:0/1012935388 learned_addr learned my addr 172.21.15.32:0/1012935388 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:28.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a5d9b640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80d3550 0x7f23a80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a659c640 1 -- 172.21.15.32:0/1012935388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80d3550 msgr2=0x7f23a80cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:28.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a659c640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80d3550 0x7f23a80cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a659c640 1 -- 172.21.15.32:0/1012935388 >> v1:172.21.15.32:6789/0 conn(0x7f23a80cf5d0 legacy=0x7f23a80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a659c640 1 -- 172.21.15.32:0/1012935388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80cdf80 con 0x7f23a815ca70 2023-10-16T08:16:28.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a559a640 1 -- 172.21.15.32:0/1012935388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477836126 0 0) 0x7f23a80c65f0 con 0x7f23a80cf5d0 2023-10-16T08:16:28.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a5d9b640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80d3550 0x7f23a80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:28.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a659c640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a80d3010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23980097e0 tx=0x7f239801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1172f676df5a76f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a559a640 1 -- 172.21.15.32:0/1012935388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980a7070 con 0x7f23a815ca70 2023-10-16T08:16:28.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a559a640 1 -- 172.21.15.32:0/1012935388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980ab430 con 0x7f23a815ca70 2023-10-16T08:16:28.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23a559a640 1 -- 172.21.15.32:0/1012935388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980ab7c0 con 0x7f23a815ca70 2023-10-16T08:16:28.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23ac89e640 1 -- 172.21.15.32:0/1012935388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 msgr2=0x7f23a80d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23ac89e640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a80d3010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23980097e0 tx=0x7f239801c430 comp rx=0 tx=0).stop 2023-10-16T08:16:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23ac89e640 1 -- 172.21.15.32:0/1012935388 shutdown_connections 2023-10-16T08:16:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23ac89e640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80d3550 0x7f23a80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23ac89e640 1 --2- 172.21.15.32:0/1012935388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a80d3010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.159+0000 7f23ac89e640 1 -- 172.21.15.32:0/1012935388 >> 172.21.15.32:0/1012935388 conn(0x7f23a809bc50 msgr2=0x7f23a80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 -- 172.21.15.32:0/1012935388 shutdown_connections 2023-10-16T08:16:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 -- 172.21.15.32:0/1012935388 wait complete. 2023-10-16T08:16:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 Processor -- start 2023-10-16T08:16:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 -- start start 2023-10-16T08:16:28.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80cf5d0 0x7f23a8152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23a80d3550 0x7f23a80b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a81534a0 con 0x7f23a815ca70 2023-10-16T08:16:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a8153780 con 0x7f23a80d3550 2023-10-16T08:16:28.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23ac89e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a8153a60 con 0x7f23a80cf5d0 2023-10-16T08:16:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50522/0 (socket says 172.21.15.32:50522) 2023-10-16T08:16:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 -- 172.21.15.32:0/3462630731 learned_addr learned my addr 172.21.15.32:0/3462630731 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a5d9b640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23a80d3550 0x7f23a80b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a659c640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80cf5d0 0x7f23a8152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 -- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80cf5d0 msgr2=0x7f23a8152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80cf5d0 0x7f23a8152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 -- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23a80d3550 msgr2=0x7f23a80b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23a80d3550 0x7f23a80b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 -- 172.21.15.32:0/3462630731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980093c0 con 0x7f23a815ca70 2023-10-16T08:16:28.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.163+0000 7f23a6d9d640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a815a220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23a0001570 tx=0x7f23a000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000a640 con 0x7f23a815ca70 2023-10-16T08:16:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a000a7c0 con 0x7f23a815ca70 2023-10-16T08:16:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000ab70 con 0x7f23a815ca70 2023-10-16T08:16:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a8153d40 con 0x7f23a815ca70 2023-10-16T08:16:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a8153240 con 0x7f23a815ca70 2023-10-16T08:16:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a8154260 con 0x7f23a815ca70 2023-10-16T08:16:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23a000acd0 con 0x7f23a815ca70 2023-10-16T08:16:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f2396ffd640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23880664a0 0x7f2388068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23a00836b0 con 0x7f23a815ca70 2023-10-16T08:16:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.167+0000 7f23a659c640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23880664a0 0x7f2388068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.171+0000 7f23a659c640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23880664a0 0x7f2388068960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23980af000 tx=0x7f2398008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.171+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23a8154260 con 0x7f23a815ca70 2023-10-16T08:16:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.359+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a8154260 con 0x7f23a815ca70 2023-10-16T08:16:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.359+0000 7f2396ffd640 1 -- 172.21.15.32:0/3462630731 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f23a8154260 con 0x7f23a815ca70 2023-10-16T08:16:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:28.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:27.396004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-16T08:16:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 7 2023-10-16T08:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23880664a0 msgr2=0x7f2388068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23880664a0 0x7f2388068960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23980af000 tx=0x7f2398008040 comp rx=0 tx=0).stop 2023-10-16T08:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 msgr2=0x7f23a815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a815a220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23a0001570 tx=0x7f23a000cca0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 shutdown_connections 2023-10-16T08:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23a815ca70 0x7f23a815a220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23a80d3550 0x7f23a80b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23880664a0 0x7f2388068960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 --2- 172.21.15.32:0/3462630731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23a80cf5d0 0x7f23a8152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 >> 172.21.15.32:0/3462630731 conn(0x7f23a809bc50 msgr2=0x7f23a80c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 shutdown_connections 2023-10-16T08:16:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.363+0000 7f23ac89e640 1 -- 172.21.15.32:0/3462630731 wait complete. 2023-10-16T08:16:28.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.523+0000 7fb454c6b640 1 Processor -- start 2023-10-16T08:16:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.523+0000 7fb454c6b640 1 -- start start 2023-10-16T08:16:28.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb454c6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb45015aa30 0x7fb4500b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb454c6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb4500b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb454c6b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45015ae10 con 0x7fb4500cfb80 2023-10-16T08:16:28.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb454c6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4500cb9e0 con 0x7fb45015aa30 2023-10-16T08:16:28.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb454c6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500b3ad0 con 0x7fb45015e4b0 2023-10-16T08:16:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb45015aa30 0x7fb4500b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb45015aa30 0x7fb4500b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50524/0 (socket says 172.21.15.32:50524) 2023-10-16T08:16:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 -- 172.21.15.32:0/2206159344 learned_addr learned my addr 172.21.15.32:0/2206159344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb447fff640 1 --2- 172.21.15.32:0/2206159344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb4500b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 -- 172.21.15.32:0/2206159344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 msgr2=0x7fb4500b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 --2- 172.21.15.32:0/2206159344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb4500b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 -- 172.21.15.32:0/2206159344 >> v1:172.21.15.186:6789/0 conn(0x7fb4500cfb80 legacy=0x7fb4500cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.527+0000 7fb44e59c640 1 -- 172.21.15.32:0/2206159344 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45015ea30 con 0x7fb45015aa30 2023-10-16T08:16:28.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb44e59c640 1 --2- 172.21.15.32:0/2206159344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb45015aa30 0x7fb4500b3590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb440009870 tx=0x7fb44001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2114836e8f302eac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb44dd9b640 1 -- 172.21.15.32:0/2206159344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400ad640 con 0x7fb45015aa30 2023-10-16T08:16:28.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb44dd9b640 1 -- 172.21.15.32:0/2206159344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4400ad7a0 con 0x7fb45015aa30 2023-10-16T08:16:28.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb44dd9b640 1 -- 172.21.15.32:0/2206159344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400adb30 con 0x7fb45015aa30 2023-10-16T08:16:28.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- 172.21.15.32:0/2206159344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb45015aa30 msgr2=0x7fb4500b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 --2- 172.21.15.32:0/2206159344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb45015aa30 0x7fb4500b3590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb440009870 tx=0x7fb44001c480 comp rx=0 tx=0).stop 2023-10-16T08:16:28.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- 172.21.15.32:0/2206159344 shutdown_connections 2023-10-16T08:16:28.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 --2- 172.21.15.32:0/2206159344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb4500b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 --2- 172.21.15.32:0/2206159344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb45015aa30 0x7fb4500b3590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- 172.21.15.32:0/2206159344 >> 172.21.15.32:0/2206159344 conn(0x7fb4500cd1c0 msgr2=0x7fb4500cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:28.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- 172.21.15.32:0/2206159344 shutdown_connections 2023-10-16T08:16:28.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- 172.21.15.32:0/2206159344 wait complete. 2023-10-16T08:16:28.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 Processor -- start 2023-10-16T08:16:28.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- start start 2023-10-16T08:16:28.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 0x7fb450154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb45015aa30 0x7fb4501f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb45015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500c5bc0 con 0x7fb4500cfb80 2023-10-16T08:16:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501553d0 con 0x7fb45015aa30 2023-10-16T08:16:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.531+0000 7fb454c6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501f9790 con 0x7fb45015e4b0 2023-10-16T08:16:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 0x7fb450154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 0x7fb450154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50534/0 (socket says 172.21.15.32:50534) 2023-10-16T08:16:28.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 -- 172.21.15.32:0/1390422411 learned_addr learned my addr 172.21.15.32:0/1390422411 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:28.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb447fff640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb45015aa30 0x7fb4501f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44ed9d640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb45015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 -- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 msgr2=0x7fb45015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb45015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 -- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb45015aa30 msgr2=0x7fb4501f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb45015aa30 0x7fb4501f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 -- 172.21.15.32:0/1390422411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400094d0 con 0x7fb4500cfb80 2023-10-16T08:16:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44ed9d640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb45015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:28.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb44e59c640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 0x7fb450154e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4400b0040 tx=0x7fb440019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b8670 con 0x7fb4500cfb80 2023-10-16T08:16:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4400a6040 con 0x7fb4500cfb80 2023-10-16T08:16:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4501f9a70 con 0x7fb4500cfb80 2023-10-16T08:16:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b89f0 con 0x7fb4500cfb80 2023-10-16T08:16:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4500c57e0 con 0x7fb4500cfb80 2023-10-16T08:16:28.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4500c5d00 con 0x7fb4500cfb80 2023-10-16T08:16:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb4400b4070 con 0x7fb4500cfb80 2023-10-16T08:16:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.535+0000 7fb446ffd640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4300664a0 0x7fb430068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.539+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb44011ea40 con 0x7fb4500cfb80 2023-10-16T08:16:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.539+0000 7fb447fff640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4300664a0 0x7fb430068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.539+0000 7fb447fff640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4300664a0 0x7fb430068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4501f9fb0 tx=0x7fb438008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.539+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb4500c5d00 con 0x7fb4500cfb80 2023-10-16T08:16:28.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.703+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4500c5d00 con 0x7fb4500cfb80 2023-10-16T08:16:28.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.703+0000 7fb446ffd640 1 -- 172.21.15.32:0/1390422411 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fb4500c5d00 con 0x7fb4500cfb80 2023-10-16T08:16:28.709 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:28.709 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4300664a0 msgr2=0x7fb430068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4300664a0 0x7fb430068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4501f9fb0 tx=0x7fb438008040 comp rx=0 tx=0).stop 2023-10-16T08:16:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 msgr2=0x7fb450154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 0x7fb450154e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4400b0040 tx=0x7fb440019220 comp rx=0 tx=0).stop 2023-10-16T08:16:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 shutdown_connections 2023-10-16T08:16:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4300664a0 0x7fb430068960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb45015e4b0 0x7fb45015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb45015aa30 0x7fb4501f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 --2- 172.21.15.32:0/1390422411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4500cfb80 0x7fb450154e90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 >> 172.21.15.32:0/1390422411 conn(0x7fb4500cd1c0 msgr2=0x7fb4500cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 shutdown_connections 2023-10-16T08:16:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.707+0000 7fb454c6b640 1 -- 172.21.15.32:0/1390422411 wait complete. 2023-10-16T08:16:28.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 Processor -- start 2023-10-16T08:16:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 -- start start 2023-10-16T08:16:28.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6e3415cb90 0x7f6e340cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e340d3010 0x7f6e340d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e340cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340d1890 con 0x7f6e340d3010 2023-10-16T08:16:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340ce770 con 0x7f6e3415cb90 2023-10-16T08:16:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e39d35640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340d0600 con 0x7f6e340d1400 2023-10-16T08:16:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e340d3010 0x7f6e340d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e340d3010 0x7f6e340d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50536/0 (socket says 172.21.15.32:50536) 2023-10-16T08:16:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2401966908 learned_addr learned my addr 172.21.15.32:0/2401966908 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e337fe640 1 --2- 172.21.15.32:0/2401966908 >> v2:172.21.15.186:3300/0 conn(0x7f6e3415cb90 0x7f6e340cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e33fff640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e340cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2401966908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 msgr2=0x7f6e340cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:28.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e340cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2401966908 >> v2:172.21.15.186:3300/0 conn(0x7f6e3415cb90 msgr2=0x7f6e340cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:28.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 --2- 172.21.15.32:0/2401966908 >> v2:172.21.15.186:3300/0 conn(0x7f6e3415cb90 0x7f6e340cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2401966908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340c6450 con 0x7f6e340d3010 2023-10-16T08:16:28.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e337fe640 1 --2- 172.21.15.32:0/2401966908 >> v2:172.21.15.186:3300/0 conn(0x7f6e3415cb90 0x7f6e340cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:28.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.871+0000 7f6e33fff640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e340cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e32ffd640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e340d0ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e3408ba70 tx=0x7f6e2801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24173de3d363f694 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e327fc640 1 -- 172.21.15.32:0/2401966908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280ab600 con 0x7f6e340d3010 2023-10-16T08:16:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e327fc640 1 -- 172.21.15.32:0/2401966908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280ab760 con 0x7f6e340d3010 2023-10-16T08:16:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e327fc640 1 -- 172.21.15.32:0/2401966908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280abaf0 con 0x7f6e340d3010 2023-10-16T08:16:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 -- 172.21.15.32:0/2401966908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 msgr2=0x7f6e340d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e340d0ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e3408ba70 tx=0x7f6e2801c440 comp rx=0 tx=0).stop 2023-10-16T08:16:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 -- 172.21.15.32:0/2401966908 shutdown_connections 2023-10-16T08:16:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e340cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2401966908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e340d0ec0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2401966908 >> v2:172.21.15.186:3300/0 conn(0x7f6e3415cb90 0x7f6e340cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 -- 172.21.15.32:0/2401966908 >> 172.21.15.32:0/2401966908 conn(0x7f6e3409bc50 msgr2=0x7f6e340d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 -- 172.21.15.32:0/2401966908 shutdown_connections 2023-10-16T08:16:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 -- 172.21.15.32:0/2401966908 wait complete. 2023-10-16T08:16:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.875+0000 7f6e39d35640 1 Processor -- start 2023-10-16T08:16:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- start start 2023-10-16T08:16:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e341524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e3414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e3415cb90 0x7f6e34151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3406c2e0 con 0x7f6e340d3010 2023-10-16T08:16:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415c0d0 con 0x7f6e3415cb90 2023-10-16T08:16:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3406b6d0 con 0x7f6e340d1400 2023-10-16T08:16:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e3414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e3414fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50540/0 (socket says 172.21.15.32:50540) 2023-10-16T08:16:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2382609893 learned_addr learned my addr 172.21.15.32:0/2382609893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e337fe640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e341524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e33fff640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e3415cb90 0x7f6e34151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 msgr2=0x7f6e341524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e341524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e3415cb90 msgr2=0x7f6e34151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e3415cb90 0x7f6e34151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 -- 172.21.15.32:0/2382609893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280094d0 con 0x7f6e340d3010 2023-10-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e337fe640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e341524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e33fff640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e3415cb90 0x7f6e34151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e32ffd640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e3414fa10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e2801aaf0 tx=0x7f6e2801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280b4800 con 0x7f6e340d3010 2023-10-16T08:16:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280b4960 con 0x7f6e340d3010 2023-10-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280b4d10 con 0x7f6e340d3010 2023-10-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3406b9b0 con 0x7f6e340d3010 2023-10-16T08:16:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3406bf70 con 0x7f6e340d3010 2023-10-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.879+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e34074010 con 0x7f6e340d3010 2023-10-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.883+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e280b3380 con 0x7f6e340d3010 2023-10-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.883+0000 7f6e1bfff640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6e200664a0 0x7f6e20068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.883+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6e2811e800 con 0x7f6e340d3010 2023-10-16T08:16:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.883+0000 7f6e337fe640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6e200664a0 0x7f6e20068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.883+0000 7f6e337fe640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6e200664a0 0x7f6e20068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400b840 tx=0x7f6e24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:28.887+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e280eb910 con 0x7f6e340d3010 2023-10-16T08:16:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e340b8740 con 0x7f6e340d3010 2023-10-16T08:16:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e1bfff640 1 -- 172.21.15.32:0/2382609893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f6e280eef30 con 0x7f6e340d3010 2023-10-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:29.052 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6e200664a0 msgr2=0x7f6e20068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6e200664a0 0x7f6e20068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400b840 tx=0x7f6e24008040 comp rx=0 tx=0).stop 2023-10-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 msgr2=0x7f6e3414fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e3414fa10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e2801aaf0 tx=0x7f6e2801ac80 comp rx=0 tx=0).stop 2023-10-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.047+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 shutdown_connections 2023-10-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e3415cb90 0x7f6e34151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340d3010 0x7f6e3414fa10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6e200664a0 0x7f6e20068960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 --2- 172.21.15.32:0/2382609893 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6e340d1400 0x7f6e341524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 >> 172.21.15.32:0/2382609893 conn(0x7f6e3409bc50 msgr2=0x7f6e340d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 shutdown_connections 2023-10-16T08:16:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.051+0000 7f6e39d35640 1 -- 172.21.15.32:0/2382609893 wait complete. 2023-10-16T08:16:29.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:29.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 Processor -- start 2023-10-16T08:16:29.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 -- start start 2023-10-16T08:16:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f72dc15cb90 0x7f72dc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72dc0d15d0 0x7f72dc0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc0ce770 con 0x7f72dc0d3010 2023-10-16T08:16:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0d0600 con 0x7f72dc0d15d0 2023-10-16T08:16:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e2e68640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0c6450 con 0x7f72dc15cb90 2023-10-16T08:16:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72dc0d15d0 0x7f72dc0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72dc0d15d0 0x7f72dc0c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50548/0 (socket says 172.21.15.32:50548) 2023-10-16T08:16:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 -- 172.21.15.32:0/1627752815 learned_addr learned my addr 172.21.15.32:0/1627752815 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e0c04640 1 --2- 172.21.15.32:0/1627752815 >> v2:172.21.15.186:3300/0 conn(0x7f72dc15cb90 0x7f72dc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 -- 172.21.15.32:0/1627752815 >> v2:172.21.15.186:3300/0 conn(0x7f72dc15cb90 msgr2=0x7f72dc0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 --2- 172.21.15.32:0/1627752815 >> v2:172.21.15.186:3300/0 conn(0x7f72dc15cb90 0x7f72dc0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 -- 172.21.15.32:0/1627752815 >> v1:172.21.15.32:6789/0 conn(0x7f72dc0d3010 legacy=0x7f72dc0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72e1405640 1 -- 172.21.15.32:0/1627752815 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc0d08a0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.227+0000 7f72db7fe640 1 -- 172.21.15.32:0/1627752815 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641945592 0 0) 0x7f72dc0ce770 con 0x7f72dc0d3010 2023-10-16T08:16:29.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e1405640 1 --2- 172.21.15.32:0/1627752815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0c6c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72dc08ba70 tx=0x7f72d401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9271c82a0e53bbe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:29.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72db7fe640 1 -- 172.21.15.32:0/1627752815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d40a57d0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72db7fe640 1 -- 172.21.15.32:0/1627752815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d40a5930 con 0x7f72dc0d15d0 2023-10-16T08:16:29.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72db7fe640 1 -- 172.21.15.32:0/1627752815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d40a5cc0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 -- 172.21.15.32:0/1627752815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 msgr2=0x7f72dc0c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 --2- 172.21.15.32:0/1627752815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0c6c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72dc08ba70 tx=0x7f72d401c910 comp rx=0 tx=0).stop 2023-10-16T08:16:29.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 -- 172.21.15.32:0/1627752815 shutdown_connections 2023-10-16T08:16:29.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 --2- 172.21.15.32:0/1627752815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0c6c10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 --2- 172.21.15.32:0/1627752815 >> v2:172.21.15.186:3300/0 conn(0x7f72dc15cb90 0x7f72dc0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 -- 172.21.15.32:0/1627752815 >> 172.21.15.32:0/1627752815 conn(0x7f72dc09bc50 msgr2=0x7f72dc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:29.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 -- 172.21.15.32:0/1627752815 shutdown_connections 2023-10-16T08:16:29.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 -- 172.21.15.32:0/1627752815 wait complete. 2023-10-16T08:16:29.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.231+0000 7f72e2e68640 1 Processor -- start 2023-10-16T08:16:29.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- start start 2023-10-16T08:16:29.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72dc0d3010 0x7f72dc150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72dc15cb90 0x7f72dc0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0cc760 con 0x7f72dc0d15d0 2023-10-16T08:16:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0caa50 con 0x7f72dc15cb90 2023-10-16T08:16:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0cad30 con 0x7f72dc0d3010 2023-10-16T08:16:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50552/0 (socket says 172.21.15.32:50552) 2023-10-16T08:16:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 -- 172.21.15.32:0/3105096448 learned_addr learned my addr 172.21.15.32:0/3105096448 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72dbfff640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72dc0d3010 0x7f72dc150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e1405640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72dc15cb90 0x7f72dc0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 -- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72dc0d3010 msgr2=0x7f72dc150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72dc0d3010 0x7f72dc150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 -- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72dc15cb90 msgr2=0x7f72dc0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72dc15cb90 0x7f72dc0cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 -- 172.21.15.32:0/3105096448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d4009b30 con 0x7f72dc0d15d0 2023-10-16T08:16:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e0c04640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0ca3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72d000d7d0 tx=0x7f72d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000b6c0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000b820 con 0x7f72dc0d15d0 2023-10-16T08:16:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000bbd0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc0cb010 con 0x7f72dc0d15d0 2023-10-16T08:16:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc0cb5d0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.235+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc0cb760 con 0x7f72dc0d15d0 2023-10-16T08:16:29.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.239+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f72d000bd30 con 0x7f72dc0d15d0 2023-10-16T08:16:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.239+0000 7f72d97fa640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72c00664d0 0x7f72c0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.239+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f72d00844b0 con 0x7f72dc0d15d0 2023-10-16T08:16:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.239+0000 7f72dbfff640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72c00664d0 0x7f72c0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.239+0000 7f72dbfff640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72c00664d0 0x7f72c0068990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72dc157390 tx=0x7f72c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.243+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f72d0055170 con 0x7f72dc0d15d0 2023-10-16T08:16:29.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72dc074010 con 0x7f72dc0d15d0 2023-10-16T08:16:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72d97fa640 1 -- 172.21.15.32:0/3105096448 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f72d0004650 con 0x7f72dc0d15d0 2023-10-16T08:16:29.435 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:29.435 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:27.396004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-16T08:16:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 7 2023-10-16T08:16:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72c00664d0 msgr2=0x7f72c0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72c00664d0 0x7f72c0068990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72dc157390 tx=0x7f72c8009040 comp rx=0 tx=0).stop 2023-10-16T08:16:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 msgr2=0x7f72dc0ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0ca3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72d000d7d0 tx=0x7f72d000dc90 comp rx=0 tx=0).stop 2023-10-16T08:16:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 shutdown_connections 2023-10-16T08:16:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72dc15cb90 0x7f72dc0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72c00664d0 0x7f72c0068990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72dc0d3010 0x7f72dc150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 --2- 172.21.15.32:0/3105096448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72dc0d15d0 0x7f72dc0ca3e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 >> 172.21.15.32:0/3105096448 conn(0x7f72dc09bc50 msgr2=0x7f72dc06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:29.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 shutdown_connections 2023-10-16T08:16:29.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.431+0000 7f72e2e68640 1 -- 172.21.15.32:0/3105096448 wait complete. 2023-10-16T08:16:29.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.595+0000 7fd46229e640 1 Processor -- start 2023-10-16T08:16:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.595+0000 7fd46229e640 1 -- start start 2023-10-16T08:16:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46229e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd45c1597c0 0x7fd45c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46229e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd45c0adc10 0x7fd45c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46229e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45c0723d0 con 0x7fd45c159ba0 2023-10-16T08:16:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46229e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c0717c0 con 0x7fd45c1597c0 2023-10-16T08:16:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46229e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c071aa0 con 0x7fd45c0adc10 2023-10-16T08:16:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd45c1597c0 0x7fd45c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd45c159ba0 0x7fd45c0cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40894/0 (socket says 172.21.15.32:40894) 2023-10-16T08:16:29.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45b7fe640 1 -- 172.21.15.32:0/2678052218 learned_addr learned my addr 172.21.15.32:0/2678052218 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:29.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd45c1597c0 0x7fd45c0ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50566/0 (socket says 172.21.15.32:50566) 2023-10-16T08:16:29.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46083b640 1 --2- 172.21.15.32:0/2678052218 >> v2:172.21.15.186:3300/0 conn(0x7fd45c0adc10 0x7fd45c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45bfff640 1 -- 172.21.15.32:0/2678052218 >> v2:172.21.15.186:3300/0 conn(0x7fd45c0adc10 msgr2=0x7fd45c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:29.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45bfff640 1 --2- 172.21.15.32:0/2678052218 >> v2:172.21.15.186:3300/0 conn(0x7fd45c0adc10 0x7fd45c0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45bfff640 1 -- 172.21.15.32:0/2678052218 >> v1:172.21.15.32:6789/0 conn(0x7fd45c159ba0 legacy=0x7fd45c0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45affd640 1 -- 172.21.15.32:0/2678052218 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876167090 0 0) 0x7fd45c0723d0 con 0x7fd45c159ba0 2023-10-16T08:16:29.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd45bfff640 1 -- 172.21.15.32:0/2678052218 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c071d80 con 0x7fd45c1597c0 2023-10-16T08:16:29.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.599+0000 7fd46083b640 1 --2- 172.21.15.32:0/2678052218 >> v2:172.21.15.186:3300/0 conn(0x7fd45c0adc10 0x7fd45c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:29.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd45bfff640 1 --2- 172.21.15.32:0/2678052218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c1597c0 0x7fd45c0ad6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd45c072340 tx=0x7fd44c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a3872fe3760125 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:29.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd45affd640 1 -- 172.21.15.32:0/2678052218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0ab600 con 0x7fd45c1597c0 2023-10-16T08:16:29.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd45affd640 1 -- 172.21.15.32:0/2678052218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0ab760 con 0x7fd45c1597c0 2023-10-16T08:16:29.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd45affd640 1 -- 172.21.15.32:0/2678052218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0abaf0 con 0x7fd45c1597c0 2023-10-16T08:16:29.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 -- 172.21.15.32:0/2678052218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c1597c0 msgr2=0x7fd45c0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 --2- 172.21.15.32:0/2678052218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c1597c0 0x7fd45c0ad6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd45c072340 tx=0x7fd44c01c460 comp rx=0 tx=0).stop 2023-10-16T08:16:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 -- 172.21.15.32:0/2678052218 shutdown_connections 2023-10-16T08:16:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 --2- 172.21.15.32:0/2678052218 >> v2:172.21.15.186:3300/0 conn(0x7fd45c0adc10 0x7fd45c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 --2- 172.21.15.32:0/2678052218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c1597c0 0x7fd45c0ad6d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 -- 172.21.15.32:0/2678052218 >> 172.21.15.32:0/2678052218 conn(0x7fd45c06c190 msgr2=0x7fd45c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:29.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 -- 172.21.15.32:0/2678052218 shutdown_connections 2023-10-16T08:16:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 -- 172.21.15.32:0/2678052218 wait complete. 2023-10-16T08:16:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 Processor -- start 2023-10-16T08:16:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.603+0000 7fd46229e640 1 -- start start 2023-10-16T08:16:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 0x7fd45c149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45c1597c0 0x7fd45c1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd45c159ba0 0x7fd45c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c156d90 con 0x7fd45c0adc10 2023-10-16T08:16:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c157070 con 0x7fd45c1597c0 2023-10-16T08:16:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c157350 con 0x7fd45c159ba0 2023-10-16T08:16:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 0x7fd45c149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 0x7fd45c149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50578/0 (socket says 172.21.15.32:50578) 2023-10-16T08:16:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 -- 172.21.15.32:0/3647121757 learned_addr learned my addr 172.21.15.32:0/3647121757 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45b7fe640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45c1597c0 0x7fd45c1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46083b640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd45c159ba0 0x7fd45c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 -- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd45c159ba0 msgr2=0x7fd45c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd45c159ba0 0x7fd45c0c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 -- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45c1597c0 msgr2=0x7fd45c1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45c1597c0 0x7fd45c1491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 -- 172.21.15.32:0/3647121757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c009420 con 0x7fd45c0adc10 2023-10-16T08:16:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd45bfff640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 0x7fd45c149b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd44c019070 tx=0x7fd44c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0b5810 con 0x7fd45c0adc10 2023-10-16T08:16:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0b5990 con 0x7fd45c0adc10 2023-10-16T08:16:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0b5d40 con 0x7fd45c0adc10 2023-10-16T08:16:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c157630 con 0x7fd45c0adc10 2023-10-16T08:16:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c156a20 con 0x7fd45c0adc10 2023-10-16T08:16:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c074010 con 0x7fd45c0adc10 2023-10-16T08:16:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.607+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd44c0b4390 con 0x7fd45c0adc10 2023-10-16T08:16:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.611+0000 7fd458ff9640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4480664a0 0x7fd448068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.611+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd44c11eaf0 con 0x7fd45c0adc10 2023-10-16T08:16:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.611+0000 7fd45b7fe640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4480664a0 0x7fd448068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.611+0000 7fd45b7fe640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4480664a0 0x7fd448068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd45c1f8660 tx=0x7fd450007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.615+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd44c0ebd20 con 0x7fd45c0adc10 2023-10-16T08:16:29.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.771+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd45c0cd3f0 con 0x7fd45c0adc10 2023-10-16T08:16:29.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.771+0000 7fd458ff9640 1 -- 172.21.15.32:0/3647121757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fd44c0ef340 con 0x7fd45c0adc10 2023-10-16T08:16:29.778 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:29.778 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:29.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4480664a0 msgr2=0x7fd448068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4480664a0 0x7fd448068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd45c1f8660 tx=0x7fd450007df0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 msgr2=0x7fd45c149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 0x7fd45c149b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd44c019070 tx=0x7fd44c01aca0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 shutdown_connections 2023-10-16T08:16:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4480664a0 0x7fd448068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd45c159ba0 0x7fd45c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd45c1597c0 0x7fd45c1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 --2- 172.21.15.32:0/3647121757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd45c0adc10 0x7fd45c149b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:29.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 >> 172.21.15.32:0/3647121757 conn(0x7fd45c06c190 msgr2=0x7fd45c0c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:29.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 shutdown_connections 2023-10-16T08:16:29.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:29.775+0000 7fd46229e640 1 -- 172.21.15.32:0/3647121757 wait complete. 2023-10-16T08:16:30.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T08:16:22.323436+0000', 'modified': '2023-10-16T08:16:27.396004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4311': {'gid': 4311, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1428800105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1428800105}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1428800105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-16T08:16:30.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T08:16:30.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T08:16:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 Processor -- start 2023-10-16T08:16:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 -- start start 2023-10-16T08:16:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 0x7f982406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f982415ce30 0x7f98240cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98241607e0 con 0x7f98240b21c0 2023-10-16T08:16:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98240cf060 con 0x7f982415ce30 2023-10-16T08:16:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982906b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9824070210 con 0x7f982415ca50 2023-10-16T08:16:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982259c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f982415ce30 0x7f98240cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 0x7f982406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 0x7f982406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:42322/0 (socket says 172.21.15.32:42322) 2023-10-16T08:16:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 -- 172.21.15.32:0/775027234 learned_addr learned my addr 172.21.15.32:0/775027234 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f982359e640 1 --1- 172.21.15.32:0/775027234 >> v1:172.21.15.186:6789/0 conn(0x7f98240b21c0 0x7f9824160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:49518/0 (socket says 172.21.15.32:49518) 2023-10-16T08:16:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9821d9b640 1 -- 172.21.15.32:0/775027234 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075718564 0 0) 0x7f98241607e0 con 0x7f98240b21c0 2023-10-16T08:16:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 -- 172.21.15.32:0/775027234 >> v2:172.21.15.186:3300/0 conn(0x7f982415ce30 msgr2=0x7f98240cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 --2- 172.21.15.32:0/775027234 >> v2:172.21.15.186:3300/0 conn(0x7f982415ce30 0x7f98240cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 -- 172.21.15.32:0/775027234 >> v1:172.21.15.186:6789/0 conn(0x7f98240b21c0 legacy=0x7f9824160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.343+0000 7f9822d9d640 1 -- 172.21.15.32:0/775027234 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240704f0 con 0x7f982415ca50 2023-10-16T08:16:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f9822d9d640 1 --2- 172.21.15.32:0/775027234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 0x7f982406c980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9814009870 tx=0x7f981401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fba51c6e3b5d0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f9821d9b640 1 -- 172.21.15.32:0/775027234 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140ab620 con 0x7f982415ca50 2023-10-16T08:16:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f9821d9b640 1 -- 172.21.15.32:0/775027234 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140ab780 con 0x7f982415ca50 2023-10-16T08:16:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f9821d9b640 1 -- 172.21.15.32:0/775027234 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140abb10 con 0x7f982415ca50 2023-10-16T08:16:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 -- 172.21.15.32:0/775027234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 msgr2=0x7f982406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 --2- 172.21.15.32:0/775027234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 0x7f982406c980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9814009870 tx=0x7f981401c510 comp rx=0 tx=0).stop 2023-10-16T08:16:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 -- 172.21.15.32:0/775027234 shutdown_connections 2023-10-16T08:16:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 --2- 172.21.15.32:0/775027234 >> v2:172.21.15.186:3300/0 conn(0x7f982415ce30 0x7f98240cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 --2- 172.21.15.32:0/775027234 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ca50 0x7f982406c980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 -- 172.21.15.32:0/775027234 >> 172.21.15.32:0/775027234 conn(0x7f98240cd1c0 msgr2=0x7f98240cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 -- 172.21.15.32:0/775027234 shutdown_connections 2023-10-16T08:16:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.347+0000 7f982906b640 1 -- 172.21.15.32:0/775027234 wait complete. 2023-10-16T08:16:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 Processor -- start 2023-10-16T08:16:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 -- start start 2023-10-16T08:16:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98240b21c0 0x7f9824151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 0x7f982414bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ce30 0x7f9824157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9824150f20 con 0x7f982415ca50 2023-10-16T08:16:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9824151880 con 0x7f98240b21c0 2023-10-16T08:16:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98241502a0 con 0x7f982415ce30 2023-10-16T08:16:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 0x7f982414bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 0x7f982414bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50586/0 (socket says 172.21.15.32:50586) 2023-10-16T08:16:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 -- 172.21.15.32:0/597045345 learned_addr learned my addr 172.21.15.32:0/597045345 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982359e640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ce30 0x7f9824157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f9822d9d640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98240b21c0 0x7f9824151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 -- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ce30 msgr2=0x7f9824157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ce30 0x7f9824157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 -- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98240b21c0 msgr2=0x7f9824151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98240b21c0 0x7f9824151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 -- 172.21.15.32:0/597045345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140094d0 con 0x7f982415ca50 2023-10-16T08:16:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982259c640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 0x7f982414bfb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f981800e870 tx=0x7f981800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f980b7fe640 1 -- 172.21.15.32:0/597045345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818009510 con 0x7f982415ca50 2023-10-16T08:16:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f980b7fe640 1 -- 172.21.15.32:0/597045345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981800eec0 con 0x7f982415ca50 2023-10-16T08:16:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f980b7fe640 1 -- 172.21.15.32:0/597045345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981800cb40 con 0x7f982415ca50 2023-10-16T08:16:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98240b2b60 con 0x7f982415ca50 2023-10-16T08:16:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.351+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9824150860 con 0x7f982415ca50 2023-10-16T08:16:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.355+0000 7f980b7fe640 1 -- 172.21.15.32:0/597045345 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f981800cca0 con 0x7f982415ca50 2023-10-16T08:16:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.355+0000 7f980b7fe640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98100664d0 0x7f9810068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.355+0000 7f980b7fe640 1 -- 172.21.15.32:0/597045345 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9818084280 con 0x7f982415ca50 2023-10-16T08:16:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.355+0000 7f982c3ef000 1 Processor -- start 2023-10-16T08:16:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.355+0000 7f9822d9d640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98100664d0 0x7f9810068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.355+0000 7f982c3ef000 1 -- start start 2023-10-16T08:16:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 0x560a0bb94c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb4a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb4b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b93fcc0 con 0x560a0bb948a0 2023-10-16T08:16:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b9a2920 con 0x560a0bb4ac50 2023-10-16T08:16:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0b99fe50 con 0x560a0bb951c0 2023-10-16T08:16:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98100664d0 0x7f9810068990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98140af000 tx=0x7f98140b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 0x560a0bb94c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 0x560a0bb94c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50594/0 (socket says 172.21.15.32:50594) 2023-10-16T08:16:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 -- 172.21.15.32:0/3303425781 learned_addr learned my addr 172.21.15.32:0/3303425781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982259c640 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb4a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982359e640 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb4b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 -- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 msgr2=0x560a0bb4a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb4a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 -- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 msgr2=0x560a0bb4b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb4b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 -- 172.21.15.32:0/3303425781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a0b694890 con 0x560a0bb948a0 2023-10-16T08:16:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9822d9d640 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 0x560a0bb94c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f981401a140 tx=0x7f9814004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16094db8a1763916 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9808ff9640 1 -- 172.21.15.32:0/3303425781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140abe40 con 0x560a0bb948a0 2023-10-16T08:16:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9808ff9640 1 -- 172.21.15.32:0/3303425781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140c0960 con 0x560a0bb948a0 2023-10-16T08:16:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f9808ff9640 1 -- 172.21.15.32:0/3303425781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140c0cd0 con 0x560a0bb948a0 2023-10-16T08:16:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- 172.21.15.32:0/3303425781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 msgr2=0x560a0bb94c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 0x560a0bb94c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f981401a140 tx=0x7f9814004740 comp rx=0 tx=0).stop 2023-10-16T08:16:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- 172.21.15.32:0/3303425781 shutdown_connections 2023-10-16T08:16:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb4b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb4a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 --2- 172.21.15.32:0/3303425781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb948a0 0x560a0bb94c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- 172.21.15.32:0/3303425781 >> 172.21.15.32:0/3303425781 conn(0x560a0bb41fc0 msgr2=0x560a0bb48ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- 172.21.15.32:0/3303425781 shutdown_connections 2023-10-16T08:16:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- 172.21.15.32:0/3303425781 wait complete. 2023-10-16T08:16:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 -- start start 2023-10-16T08:16:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.359+0000 7f982c3ef000 1 Processor -- start 2023-10-16T08:16:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bb942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bb4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb951c0 0x560a0bb503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b99b740 con 0x560a0bb4ac50 2023-10-16T08:16:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b95ad10 con 0x560a0bb951c0 2023-10-16T08:16:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0b95a9b0 con 0x560a0bb948a0 2023-10-16T08:16:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 10 client.0 _ll_register_callbacks cb 0x560a0bb36390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:16:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bb942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bb4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bb942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50604/0 (socket says 172.21.15.32:50604) 2023-10-16T08:16:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 -- 172.21.15.32:0/2193460764 learned_addr learned my addr 172.21.15.32:0/2193460764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982359e640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb951c0 0x560a0bb503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 -- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 msgr2=0x560a0bb4fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bb4fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 -- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb951c0 msgr2=0x560a0bb503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb951c0 0x560a0bb503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 -- 172.21.15.32:0/2193460764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98240704f0 con 0x560a0bb4ac50 2023-10-16T08:16:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982259c640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bb4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9822d9d640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bb942f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9814000c10 tx=0x7f981401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69d799adec346e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:16:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 1 -- 172.21.15.32:0/2193460764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814004380 con 0x560a0bb4ac50 2023-10-16T08:16:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 1 -- 172.21.15.32:0/2193460764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f98140c0960 con 0x560a0bb4ac50 2023-10-16T08:16:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a0b94f370 con 0x560a0bb4ac50 2023-10-16T08:16:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 1 -- 172.21.15.32:0/2193460764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98140044e0 con 0x560a0bb4ac50 2023-10-16T08:16:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 10 client.4422 fetch_fsmap learned FSMap version 7 2023-10-16T08:16:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560a0b95d460 con 0x560a0bb4ac50 2023-10-16T08:16:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 1 -- 172.21.15.32:0/2193460764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a0b94f370 con 0x560a0bb4ac50 2023-10-16T08:16:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 1 -- 172.21.15.32:0/2193460764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f98140c0b20 con 0x560a0bb4ac50 2023-10-16T08:16:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2023-10-16T08:16:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 10 client.4422 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T08:16:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 20 client.4422 populate_metadata read hostname 'smithi032' 2023-10-16T08:16:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bb57110 0x560a0bba6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 4 client.4422 mds_command: new command op to 4311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T08:16:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 --> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a0b9352d0 con 0x560a0bb57110 2023-10-16T08:16:30.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982259c640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bb57110 0x560a0bba6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982259c640 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bb57110 0x560a0bba6950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c00d3d7c4ba8c8 server_cookie=53d204edb33c9360 in_seq=0 out_seq=0 2023-10-16T08:16:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 10 client.4422 ms_handle_connect on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 1 -- 172.21.15.32:0/2193460764 <== mds.0 v2:172.21.15.32:6834/1428800105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a0b9352d0 con 0x560a0bb57110 2023-10-16T08:16:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f9801ffb640 10 client.4422 handle_command_reply: tid=0 2023-10-16T08:16:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 client.4422 shutdown 2023-10-16T08:16:30.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 msgr2=0x560a0bb942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.363+0000 7f982c3ef000 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bb942f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9814000c10 tx=0x7f981401ad50 comp rx=0 tx=0).stop 2023-10-16T08:16:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 shutdown_connections 2023-10-16T08:16:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bb57110 0x560a0bba6950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb951c0 0x560a0bb503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bb4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 --2- 172.21.15.32:0/2193460764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bb942f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 >> 172.21.15.32:0/2193460764 conn(0x560a0bb41fc0 msgr2=0x560a0bb496f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f9801ffb640 0 client.4422 ms_handle_reset on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 shutdown_connections 2023-10-16T08:16:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 1 -- 172.21.15.32:0/2193460764 wait complete. 2023-10-16T08:16:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.367+0000 7f982c3ef000 20 client.4422 trim_cache size 0 max 16384 2023-10-16T08:16:30.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.379+0000 7f982c3ef000 1 Processor -- start 2023-10-16T08:16:30.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.379+0000 7f982c3ef000 1 -- start start 2023-10-16T08:16:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bba7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb948a0 0x560a0bba7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb8f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b655940 con 0x560a0bb4ac50 2023-10-16T08:16:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b6951b0 con 0x560a0bb948a0 2023-10-16T08:16:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0bb43a70 con 0x560a0bb951c0 2023-10-16T08:16:30.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bba7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bba7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50618/0 (socket says 172.21.15.32:50618) 2023-10-16T08:16:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 -- 172.21.15.32:0/80676191 learned_addr learned my addr 172.21.15.32:0/80676191 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982259c640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb948a0 0x560a0bba7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982359e640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb8f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 -- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 msgr2=0x560a0bb8f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb8f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 -- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb948a0 msgr2=0x560a0bba7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb948a0 0x560a0bba7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 -- 172.21.15.32:0/80676191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a0b943760 con 0x560a0bb4ac50 2023-10-16T08:16:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982259c640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb948a0 0x560a0bba7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:30.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982359e640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb8f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f9822d9d640 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bba7960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98140ad040 tx=0x7f98140d0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9882e86f92caa6b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f98097fa640 1 -- 172.21.15.32:0/80676191 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814004380 con 0x560a0bb4ac50 2023-10-16T08:16:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f98097fa640 1 -- 172.21.15.32:0/80676191 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140044e0 con 0x560a0bb4ac50 2023-10-16T08:16:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f98097fa640 1 -- 172.21.15.32:0/80676191 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140c09e0 con 0x560a0bb4ac50 2023-10-16T08:16:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- 172.21.15.32:0/80676191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 msgr2=0x560a0bba7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bba7960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98140ad040 tx=0x7f98140d0000 comp rx=0 tx=0).stop 2023-10-16T08:16:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- 172.21.15.32:0/80676191 shutdown_connections 2023-10-16T08:16:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb951c0 0x560a0bb8f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb948a0 0x560a0bba7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- 172.21.15.32:0/80676191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb4ac50 0x560a0bba7960 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- 172.21.15.32:0/80676191 >> 172.21.15.32:0/80676191 conn(0x560a0bb41fc0 msgr2=0x560a0bb560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- 172.21.15.32:0/80676191 shutdown_connections 2023-10-16T08:16:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- 172.21.15.32:0/80676191 wait complete. 2023-10-16T08:16:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- start start 2023-10-16T08:16:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 Processor -- start 2023-10-16T08:16:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bbafd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 0x560a0bbb0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b9496c0 con 0x560a0bb951c0 2023-10-16T08:16:30.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0b99a5d0 con 0x560a0bb4ac50 2023-10-16T08:16:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0b95dd30 con 0x560a0bb948a0 2023-10-16T08:16:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982c3ef000 10 client.0 _ll_register_callbacks cb 0x7f98140096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:16:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.383+0000 7f982359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 0x560a0bbb0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 0x560a0bbb0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50634/0 (socket says 172.21.15.32:50634) 2023-10-16T08:16:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 -- 172.21.15.32:0/1011795193 learned_addr learned my addr 172.21.15.32:0/1011795193 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982259c640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bbafd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9822d9d640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 -- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 msgr2=0x560a0bbafd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bbafd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 -- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 msgr2=0x560a0bb927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb927c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 -- 172.21.15.32:0/1011795193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a0b95d460 con 0x560a0bb951c0 2023-10-16T08:16:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982359e640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 0x560a0bbb0250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f981c007f60 tx=0x7f981c007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b5eef1b3e0b82e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:16:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 1 -- 172.21.15.32:0/1011795193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c017640 con 0x560a0bb951c0 2023-10-16T08:16:30.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 1 -- 172.21.15.32:0/1011795193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f981c01bb10 con 0x560a0bb951c0 2023-10-16T08:16:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a0bb4d3b0 con 0x560a0bb951c0 2023-10-16T08:16:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 1 -- 172.21.15.32:0/1011795193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f981c01bcd0 con 0x560a0bb951c0 2023-10-16T08:16:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 10 client.4428 fetch_fsmap learned FSMap version 7 2023-10-16T08:16:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560a0b694cc0 con 0x560a0bb951c0 2023-10-16T08:16:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 1 -- 172.21.15.32:0/1011795193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a0bb4d3b0 con 0x560a0bb951c0 2023-10-16T08:16:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 1 -- 172.21.15.32:0/1011795193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f981c0177a0 con 0x560a0bb951c0 2023-10-16T08:16:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2023-10-16T08:16:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 10 client.4428 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T08:16:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 20 client.4428 populate_metadata read hostname 'smithi032' 2023-10-16T08:16:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bbf9170 0x560a0bbfb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 4 client.4428 mds_command: new command op to 4311 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T08:16:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 --> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a0b9336f0 con 0x560a0bbf9170 2023-10-16T08:16:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9822d9d640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bbf9170 0x560a0bbfb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9822d9d640 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bbf9170 0x560a0bbfb560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884e557cfc66047e server_cookie=ef3d3a907caea6ce in_seq=0 out_seq=0 2023-10-16T08:16:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 10 client.4428 ms_handle_connect on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 1 -- 172.21.15.32:0/1011795193 <== mds.0 v2:172.21.15.32:6834/1428800105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560a0b9336f0 con 0x560a0bbf9170 2023-10-16T08:16:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 10 client.4428 handle_command_reply: tid=0 2023-10-16T08:16:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 client.4428 shutdown 2023-10-16T08:16:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 msgr2=0x560a0bbb0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 0x560a0bbb0250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f981c007f60 tx=0x7f981c007490 comp rx=0 tx=0).stop 2023-10-16T08:16:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 shutdown_connections 2023-10-16T08:16:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x560a0bbf9170 0x560a0bbfb560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560a0bb951c0 0x560a0bbb0250 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x560a0bb948a0 0x560a0bbafd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 --2- 172.21.15.32:0/1011795193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a0bb4ac50 0x560a0bb927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 >> 172.21.15.32:0/1011795193 conn(0x560a0bb41fc0 msgr2=0x560a0bba6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f9801ffb640 0 client.4428 ms_handle_reset on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 shutdown_connections 2023-10-16T08:16:30.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 1 -- 172.21.15.32:0/1011795193 wait complete. 2023-10-16T08:16:30.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982c3ef000 20 client.4428 trim_cache size 0 max 16384 2023-10-16T08:16:30.406 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:16:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "943c836a-5c4c-4cb7-a190-9643b3476f99", 2023-10-16T08:16:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-10-16T08:16:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4311, 2023-10-16T08:16:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-10-16T08:16:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-10-16T08:16:30.407 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-10-16T08:16:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 7.4329056089999996, 2023-10-16T08:16:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 7, 2023-10-16T08:16:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 25, 2023-10-16T08:16:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 24, 2023-10-16T08:16:30.408 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 14.388204330000001 2023-10-16T08:16:30.409 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:16:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98100664d0 msgr2=0x7f9810068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.387+0000 7f982906b640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98100664d0 0x7f9810068990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98140af000 tx=0x7f98140b6040 comp rx=0 tx=0).stop 2023-10-16T08:16:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 msgr2=0x7f982414bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 0x7f982414bfb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f981800e870 tx=0x7f981800ec70 comp rx=0 tx=0).stop 2023-10-16T08:16:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 shutdown_connections 2023-10-16T08:16:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98100664d0 0x7f9810068990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f982415ce30 0x7f9824157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f982415ca50 0x7f982414bfb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 --2- 172.21.15.32:0/597045345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98240b21c0 0x7f9824151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 >> 172.21.15.32:0/597045345 conn(0x7f98240cd1c0 msgr2=0x7f9824160c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 shutdown_connections 2023-10-16T08:16:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.391+0000 7f982906b640 1 -- 172.21.15.32:0/597045345 wait complete. 2023-10-16T08:16:30.413 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:16:30.413 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:30.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:16:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.551+0000 7f7165c46640 1 Processor -- start 2023-10-16T08:16:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.551+0000 7f7165c46640 1 -- start start 2023-10-16T08:16:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f7165c46640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71601597c0 0x7f71600ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f7165c46640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f71600adc10 0x7f71600cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f7165c46640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71600723d0 con 0x7f7160159ba0 2023-10-16T08:16:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f7165c46640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71600717c0 con 0x7f71601597c0 2023-10-16T08:16:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f7165c46640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7160071aa0 con 0x7f71600adc10 2023-10-16T08:16:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71601597c0 0x7f71600ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71601597c0 0x7f71600ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50644/0 (socket says 172.21.15.32:50644) 2023-10-16T08:16:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 -- 172.21.15.32:0/4138099220 learned_addr learned my addr 172.21.15.32:0/4138099220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715ffff640 1 --2- 172.21.15.32:0/4138099220 >> v2:172.21.15.186:3300/0 conn(0x7f71600adc10 0x7f71600cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 -- 172.21.15.32:0/4138099220 >> v2:172.21.15.186:3300/0 conn(0x7f71600adc10 msgr2=0x7f71600cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 --2- 172.21.15.32:0/4138099220 >> v2:172.21.15.186:3300/0 conn(0x7f71600adc10 0x7f71600cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 -- 172.21.15.32:0/4138099220 >> v1:172.21.15.186:6789/0 conn(0x7f7160159ba0 legacy=0x7f71600cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.555+0000 7f715f7fe640 1 -- 172.21.15.32:0/4138099220 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160071d80 con 0x7f71601597c0 2023-10-16T08:16:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f715f7fe640 1 --2- 172.21.15.32:0/4138099220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71601597c0 0x7f71600ad6d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7160072340 tx=0x7f715001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56931cc64e414448 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f715e7fc640 1 -- 172.21.15.32:0/4138099220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500a6040 con 0x7f71601597c0 2023-10-16T08:16:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f715e7fc640 1 -- 172.21.15.32:0/4138099220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71500aa600 con 0x7f71601597c0 2023-10-16T08:16:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f715e7fc640 1 -- 172.21.15.32:0/4138099220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500aa990 con 0x7f71601597c0 2023-10-16T08:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 -- 172.21.15.32:0/4138099220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71601597c0 msgr2=0x7f71600ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 --2- 172.21.15.32:0/4138099220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71601597c0 0x7f71600ad6d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7160072340 tx=0x7f715001c500 comp rx=0 tx=0).stop 2023-10-16T08:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 -- 172.21.15.32:0/4138099220 shutdown_connections 2023-10-16T08:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 --2- 172.21.15.32:0/4138099220 >> v2:172.21.15.186:3300/0 conn(0x7f71600adc10 0x7f71600cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 --2- 172.21.15.32:0/4138099220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71601597c0 0x7f71600ad6d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 -- 172.21.15.32:0/4138099220 >> 172.21.15.32:0/4138099220 conn(0x7f716006c190 msgr2=0x7f716006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 -- 172.21.15.32:0/4138099220 shutdown_connections 2023-10-16T08:16:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 -- 172.21.15.32:0/4138099220 wait complete. 2023-10-16T08:16:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 Processor -- start 2023-10-16T08:16:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.559+0000 7f7165c46640 1 -- start start 2023-10-16T08:16:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 0x7f71600c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71601597c0 0x7f71600c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7160159ba0 0x7f716015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c4cc0 con 0x7f71600adc10 2023-10-16T08:16:30.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c68c0 con 0x7f7160159ba0 2023-10-16T08:16:30.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c7a40 con 0x7f71601597c0 2023-10-16T08:16:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 0x7f71600c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 0x7f71600c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50658/0 (socket says 172.21.15.32:50658) 2023-10-16T08:16:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 -- 172.21.15.32:0/4100943294 learned_addr learned my addr 172.21.15.32:0/4100943294 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 -- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71601597c0 msgr2=0x7f71600c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715effd640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71601597c0 0x7f71600c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71601597c0 0x7f71600c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 -- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7160159ba0 msgr2=0x7f716015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7160159ba0 0x7f716015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 -- 172.21.15.32:0/4100943294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500094d0 con 0x7f71600adc10 2023-10-16T08:16:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715effd640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71601597c0 0x7f71600c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f715f7fe640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 0x7f71600c6380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f715001a130 tx=0x7f715001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500b3600 con 0x7f71600adc10 2023-10-16T08:16:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71500b3780 con 0x7f71600adc10 2023-10-16T08:16:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500b3b30 con 0x7f71600adc10 2023-10-16T08:16:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71600c7d20 con 0x7f71600adc10 2023-10-16T08:16:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71600c82e0 con 0x7f71600adc10 2023-10-16T08:16:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7160074010 con 0x7f71600adc10 2023-10-16T08:16:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.563+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f71500b3c90 con 0x7f71600adc10 2023-10-16T08:16:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.567+0000 7f7147fff640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f714c0664a0 0x7f714c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.567+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f715011e2b0 con 0x7f71600adc10 2023-10-16T08:16:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.567+0000 7f715effd640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f714c0664a0 0x7f714c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.567+0000 7f715effd640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f714c0664a0 0x7f714c068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71600cdf70 tx=0x7f7154009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.571+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f71500eb550 con 0x7f71600adc10 2023-10-16T08:16:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.759+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7160164610 con 0x7f71600adc10 2023-10-16T08:16:30.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.759+0000 7f7147fff640 1 -- 172.21.15.32:0/4100943294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f71500eeb70 con 0x7f71600adc10 2023-10-16T08:16:30.765 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T08:16:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f714c0664a0 msgr2=0x7f714c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f714c0664a0 0x7f714c068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71600cdf70 tx=0x7f7154009040 comp rx=0 tx=0).stop 2023-10-16T08:16:30.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 msgr2=0x7f71600c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 0x7f71600c6380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f715001a130 tx=0x7f715001ad40 comp rx=0 tx=0).stop 2023-10-16T08:16:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 shutdown_connections 2023-10-16T08:16:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7160159ba0 0x7f716015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f714c0664a0 0x7f714c068960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71601597c0 0x7f71600c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 --2- 172.21.15.32:0/4100943294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71600adc10 0x7f71600c6380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 >> 172.21.15.32:0/4100943294 conn(0x7f716006c190 msgr2=0x7f71600ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 shutdown_connections 2023-10-16T08:16:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:30.763+0000 7f7165c46640 1 -- 172.21.15.32:0/4100943294 wait complete. 2023-10-16T08:16:30.786 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T08:16:30.786 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:16:30.786 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:16:30.787 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-16T08:16:30.787 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:30.787 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T08:16:30.788 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:16:30.788 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:16:30.788 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-16T08:16:30.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:30.789 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:16:30.799 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:16:30.799 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:16:30.799 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:16:30.800 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:16:30.800 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:16:30.800 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:16:30.800 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:16:30.800 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:16:30.801 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:16:30.801 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:16:30.801 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:16:30.801 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:16:30.801 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:16:30.802 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:16:30.802 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:16:30.802 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:16:30.802 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42414sec preferred_lft 42414sec 2023-10-16T08:16:30.802 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:16:30.802 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:16:30.803 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:16:30.803 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:16:30.803 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:16:30.803 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:16:30.803 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:16:30.804 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:16:30.804 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:16:30.804 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T08:16:30.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:30.805 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:16:30.805 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:16:30.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-10-16T08:16:30.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-10-16T08:16:30.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-10-16T08:16:30.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T08:16:30.807 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:16:30.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:30.918 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:16:30.938 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:16:30.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:30.940 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:16:31.015 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:16:31.015 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:16:31.015 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:16:31.015 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:16:31.016 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:16:31.016 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:16:31.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.017 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:16:31.017 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:16:31.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:16:31.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:16:31.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:16:31.018 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:16:31.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.110 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:16:31.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.121 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-10-16T08:16:31.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.175 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:16:31.175 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:16:31.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:31.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-16T08:16:31.176 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:16:31.259 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-16T08:16:31.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.260 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:16:31.260 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:16:31.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-16T08:16:31.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T08:16:31.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-16T08:16:31.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-16T08:16:31.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-16T08:16:31.262 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:16:31.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.441 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:16:31.442 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:16:31.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:16:31.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T08:16:31.442 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:16:31.485 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-16T08:16:31.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:16:31.486 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:31.542 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-16T08:16:31.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:16:31.543 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:31.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:16:31.659 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:16:31.711 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:16:31.711 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:16:31.712 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:16:31.712 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:16:31.712 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:16:31.712 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:16:31.712 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:16:31.712 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:16:31.713 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:16:31.713 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:16:31.713 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:16:31.713 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:16:31.713 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:16:31.714 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:16:31.714 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:16:31.714 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:16:31.714 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:16:31.714 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:16:31.715 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:16:31.715 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:16:31.715 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:16:31.715 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:16:31.715 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:16:31.716 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:16:31.716 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:16:31.716 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:16:31.716 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:16:31.716 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:16:31.717 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:16:31.717 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:31.717 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:31.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.718 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:16:31.770 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:16:31.770 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-16T08:16:31.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:16:31.837 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:16:31.891 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:16:31.891 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:16:31.891 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:16:31.892 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:16:31.892 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:16:31.892 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:16:31.892 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:16:31.892 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:16:31.892 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:16:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:16:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:16:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:16:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:16:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:16:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:16:31.894 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:16:31.894 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:16:31.894 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:16:31.894 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:16:31.894 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:16:31.894 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:16:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:16:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:16:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:16:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:16:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:16:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:16:31.896 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:16:31.896 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:16:31.896 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:31.896 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:31.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:31.897 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:16:31.944 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:2023-10-16T08:16:31.939+0000 7fd96e248580 -1 init, newargv = 0x563816c6ea80 newargc=15 2023-10-16T08:16:31.945 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[42907]: starting ceph client 2023-10-16T08:16:31.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[42907]: starting fuse 2023-10-16T08:16:33.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:16:33.077 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:16:33.132 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:16:33.132 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:16:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:16:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:16:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:16:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:16:33.133 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:16:33.134 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:16:33.134 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:16:33.134 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:16:33.134 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:16:33.134 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:16:33.135 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:16:33.135 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:16:33.135 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:16:33.135 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:16:33.135 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:16:33.136 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:16:33.136 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:16:33.136 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:16:33.136 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:16:33.136 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:16:33.137 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:16:33.137 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:16:33.137 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:16:33.137 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:16:33.137 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:16:33.138 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:16:33.138 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:16:33.138 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:33.138 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:33.138 INFO:teuthology.orchestra.run.smithi032.stdout:/home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:33.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:33.140 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:16:33.190 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:16:33.191 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:16:33.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:16:33.192 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:16:33.192 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:16:33.192 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:16:33.193 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:16:33.193 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:16:33.193 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:16:33.193 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-16T08:16:33.194 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:16:33.194 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-16T08:16:33.195 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:16:33.195 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:16:33.195 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:16:33.195 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:16:33.196 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:16:33.196 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:16:33.197 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:16:33.197 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:16:33.197 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:16:33.198 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:16:33.198 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:16:33.199 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:16:33.199 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:16:33.199 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:16:33.200 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:16:33.200 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.admin")) 2023-10-16T08:16:33.200 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:16:33.326 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.admin.42907.asok 2023-10-16T08:16:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:16:33.339 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42907.asok 2023-10-16T08:16:33.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:33.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42907.asok status 2023-10-16T08:16:33.521 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:16:33.521 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:16:33.521 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:16:33.521 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:16:33.522 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "admin", 2023-10-16T08:16:33.522 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:16:33.522 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-16T08:16:33.522 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "42907", 2023-10-16T08:16:33.522 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:16:33.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:16:33.523 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:16:33.523 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:16:33.523 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4440, 2023-10-16T08:16:33.523 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:16:33.523 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:16:33.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:16:33.524 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 4440 2023-10-16T08:16:33.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:16:33.524 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:16:33.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:16:33.524 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:16:33.525 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1223499592 2023-10-16T08:16:33.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:16:33.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:16:33.525 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:16:33.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:16:33.526 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:16:33.526 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1223499592 2023-10-16T08:16:33.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:16:33.526 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.4440 192.168.0.1:0/1223499592", 2023-10-16T08:16:33.526 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1223499592", 2023-10-16T08:16:33.526 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:16:33.527 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 7, 2023-10-16T08:16:33.527 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 25, 2023-10-16T08:16:33.527 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:16:33.527 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:16:33.527 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2023-10-16T08:16:33.528 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:16:33.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:33.537 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:16:33.537 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:33.547 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:16:33.548 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:33.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:33.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:33.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:33.615 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-16T08:16:33.693 INFO:teuthology.orchestra.run.smithi032.stdout:# file: . 2023-10-16T08:16:33.693 INFO:teuthology.orchestra.run.smithi032.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-16T08:16:33.694 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:33.695 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:16:33.695 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:16:33.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:33.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3bd3c4640 1 Processor -- start 2023-10-16T08:16:33.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3bd3c4640 1 -- start start 2023-10-16T08:16:33.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3bd3c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3b815cb90 0x7ff3b80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:33.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3bd3c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80ce770 con 0x7ff3b80d15d0 2023-10-16T08:16:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3bd3c4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80d0600 con 0x7ff3b80d3010 2023-10-16T08:16:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3bd3c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b80c6450 con 0x7ff3b815cb90 2023-10-16T08:16:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3b815cb90 0x7ff3b80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3b80d15d0 0x7ff3b80c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33866/0 (socket says 172.21.15.32:33866) 2023-10-16T08:16:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b77fe640 1 -- 172.21.15.32:0/785685703 learned_addr learned my addr 172.21.15.32:0/785685703 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b6ffd640 1 -- 172.21.15.32:0/785685703 >> v1:172.21.15.186:6789/0 conn(0x7ff3b80d3010 legacy=0x7ff3b80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b6ffd640 1 -- 172.21.15.32:0/785685703 >> v1:172.21.15.32:6789/0 conn(0x7ff3b80d15d0 legacy=0x7ff3b80c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b6ffd640 1 -- 172.21.15.32:0/785685703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80d08a0 con 0x7ff3b815cb90 2023-10-16T08:16:33.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.927+0000 7ff3b67fc640 1 -- 172.21.15.32:0/785685703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998567255 0 0) 0x7ff3b80ce770 con 0x7ff3b80d15d0 2023-10-16T08:16:33.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3b6ffd640 1 --2- 172.21.15.32:0/785685703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cfb10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80097a0 tx=0x7ff3a801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31135183924c9026 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:33.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3b67fc640 1 -- 172.21.15.32:0/785685703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80ac600 con 0x7ff3b815cb90 2023-10-16T08:16:33.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3b67fc640 1 -- 172.21.15.32:0/785685703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a80a8050 con 0x7ff3b815cb90 2023-10-16T08:16:33.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3b67fc640 1 -- 172.21.15.32:0/785685703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a80ac980 con 0x7ff3b815cb90 2023-10-16T08:16:33.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/785685703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b815cb90 msgr2=0x7ff3b80cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:33.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/785685703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cfb10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80097a0 tx=0x7ff3a801c3a0 comp rx=0 tx=0).stop 2023-10-16T08:16:33.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/785685703 shutdown_connections 2023-10-16T08:16:33.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/785685703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cfb10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:33.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/785685703 >> 172.21.15.32:0/785685703 conn(0x7ff3b809bc50 msgr2=0x7ff3b80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:33.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/785685703 shutdown_connections 2023-10-16T08:16:33.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.931+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/785685703 wait complete. 2023-10-16T08:16:33.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 Processor -- start 2023-10-16T08:16:33.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 -- start start 2023-10-16T08:16:33.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3b80d15d0 0x7ff3b809d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:33.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 0x7ff3b8156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:33.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80a07b0 con 0x7ff3b80d3010 2023-10-16T08:16:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8158940 con 0x7ff3b815cb90 2023-10-16T08:16:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b809ff10 con 0x7ff3b80d15d0 2023-10-16T08:16:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 0x7ff3b8156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3b6ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3b80d15d0 0x7ff3b809d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 0x7ff3b8156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36118/0 (socket says 172.21.15.32:36118) 2023-10-16T08:16:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 -- 172.21.15.32:0/3157034888 learned_addr learned my addr 172.21.15.32:0/3157034888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3b77fe640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 -- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3b80d15d0 msgr2=0x7ff3b809d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3b80d15d0 0x7ff3b809d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 -- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b815cb90 msgr2=0x7ff3b80cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:33.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:33.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 -- 172.21.15.32:0/3157034888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80093b0 con 0x7ff3b80d3010 2023-10-16T08:16:33.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3affff640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 0x7ff3b8156b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c500 tx=0x7ff3a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:33.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0005220 con 0x7ff3b80d3010 2023-10-16T08:16:33.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0005380 con 0x7ff3b80d3010 2023-10-16T08:16:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0007270 con 0x7ff3b80d3010 2023-10-16T08:16:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b80a01f0 con 0x7ff3b80d3010 2023-10-16T08:16:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.935+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b809d8f0 con 0x7ff3b80d3010 2023-10-16T08:16:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.939+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b81587a0 con 0x7ff3b80d3010 2023-10-16T08:16:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.939+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff3a00073d0 con 0x7ff3b80d3010 2023-10-16T08:16:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.939+0000 7ff3af7fe640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3980664a0 0x7ff398068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.939+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff3a0083970 con 0x7ff3b80d3010 2023-10-16T08:16:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.939+0000 7ff3b6ffd640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3980664a0 0x7ff398068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.939+0000 7ff3b6ffd640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3980664a0 0x7ff398068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80ae000 tx=0x7ff3a80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:33.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:33.943+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff3b81587a0 con 0x7ff3b80d3010 2023-10-16T08:16:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.103+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3b81587a0 con 0x7ff3b80d3010 2023-10-16T08:16:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3af7fe640 1 -- 172.21.15.32:0/3157034888 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7ff3b81587a0 con 0x7ff3b80d3010 2023-10-16T08:16:34.111 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:34.111 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3980664a0 msgr2=0x7ff398068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3980664a0 0x7ff398068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80ae000 tx=0x7ff3a80a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 msgr2=0x7ff3b8156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 0x7ff3b8156b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c500 tx=0x7ff3a000c9c0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 shutdown_connections 2023-10-16T08:16:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3b815cb90 0x7ff3b80cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b80d3010 0x7ff3b8156b10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3980664a0 0x7ff398068960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 --2- 172.21.15.32:0/3157034888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3b80d15d0 0x7ff3b809d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 >> 172.21.15.32:0/3157034888 conn(0x7ff3b809bc50 msgr2=0x7ff3b806b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 shutdown_connections 2023-10-16T08:16:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.107+0000 7ff3bd3c4640 1 -- 172.21.15.32:0/3157034888 wait complete. 2023-10-16T08:16:34.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:16:34.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:16:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 Processor -- start 2023-10-16T08:16:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 -- start start 2023-10-16T08:16:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcef415cba0 0x7fcef40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef40d1670 con 0x7fcef40b84e0 2023-10-16T08:16:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40d1460 con 0x7fcef415cba0 2023-10-16T08:16:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcefb26b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40d1950 con 0x7fcef40d0210 2023-10-16T08:16:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9007640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcef415cba0 0x7fcef40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef8806640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcef40b84e0 0x7fcef40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33878/0 (socket says 172.21.15.32:33878) 2023-10-16T08:16:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef8806640 1 -- 172.21.15.32:0/2676095708 learned_addr learned my addr 172.21.15.32:0/2676095708 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9808640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9007640 1 -- 172.21.15.32:0/2676095708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 msgr2=0x7fcef40b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9007640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef40b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9007640 1 -- 172.21.15.32:0/2676095708 >> v1:172.21.15.32:6789/0 conn(0x7fcef40b84e0 legacy=0x7fcef40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9007640 1 -- 172.21.15.32:0/2676095708 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40d1c30 con 0x7fcef415cba0 2023-10-16T08:16:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.275+0000 7fcef9808640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcef9007640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef40d0f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcef00097c0 tx=0x7fcef001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0903abbd7c1f0a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fceebfff640 1 -- 172.21.15.32:0/2676095708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00ab680 con 0x7fcef415cba0 2023-10-16T08:16:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fceebfff640 1 -- 172.21.15.32:0/2676095708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef00ab7e0 con 0x7fcef415cba0 2023-10-16T08:16:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fceebfff640 1 -- 172.21.15.32:0/2676095708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00abb70 con 0x7fcef415cba0 2023-10-16T08:16:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 -- 172.21.15.32:0/2676095708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 msgr2=0x7fcef40d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef40d0f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcef00097c0 tx=0x7fcef001c480 comp rx=0 tx=0).stop 2023-10-16T08:16:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 -- 172.21.15.32:0/2676095708 shutdown_connections 2023-10-16T08:16:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2676095708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef40d0f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 -- 172.21.15.32:0/2676095708 >> 172.21.15.32:0/2676095708 conn(0x7fcef40719c0 msgr2=0x7fcef4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 -- 172.21.15.32:0/2676095708 shutdown_connections 2023-10-16T08:16:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.279+0000 7fcefb26b640 1 -- 172.21.15.32:0/2676095708 wait complete. 2023-10-16T08:16:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 Processor -- start 2023-10-16T08:16:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 -- start start 2023-10-16T08:16:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40b84e0 0x7fcef41552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef414e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415f440 con 0x7fcef415cba0 2023-10-16T08:16:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415dd30 con 0x7fcef40b84e0 2023-10-16T08:16:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415e010 con 0x7fcef40d0210 2023-10-16T08:16:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef414e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef414e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36138/0 (socket says 172.21.15.32:36138) 2023-10-16T08:16:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 -- 172.21.15.32:0/2436220614 learned_addr learned my addr 172.21.15.32:0/2436220614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9007640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40b84e0 0x7fcef41552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef8806640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 -- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 msgr2=0x7fcef414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef414dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 -- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40b84e0 msgr2=0x7fcef41552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40b84e0 0x7fcef41552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 -- 172.21.15.32:0/2436220614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0009420 con 0x7fcef415cba0 2023-10-16T08:16:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9007640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40b84e0 0x7fcef41552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef8806640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcef9808640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef414e140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fceec007ec0 tx=0x7fceec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec003ba0 con 0x7fcef415cba0 2023-10-16T08:16:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec00f040 con 0x7fcef415cba0 2023-10-16T08:16:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef415e2f0 con 0x7fcef415cba0 2023-10-16T08:16:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00de80 con 0x7fcef415cba0 2023-10-16T08:16:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.283+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef415f060 con 0x7fcef415cba0 2023-10-16T08:16:34.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.287+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef415e7a0 con 0x7fcef415cba0 2023-10-16T08:16:34.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.287+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fceec004030 con 0x7fcef415cba0 2023-10-16T08:16:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.287+0000 7fcee9ffb640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fced80664a0 0x7fced8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.287+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fceec084550 con 0x7fcef415cba0 2023-10-16T08:16:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.287+0000 7fcef9007640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fced80664a0 0x7fced8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.287+0000 7fcef9007640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fced80664a0 0x7fced8068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcef00af000 tx=0x7fcef0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.291+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcef415e7a0 con 0x7fcef415cba0 2023-10-16T08:16:34.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcef415e7a0 con 0x7fcef415cba0 2023-10-16T08:16:34.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcee9ffb640 1 -- 172.21.15.32:0/2436220614 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcef415e7a0 con 0x7fcef415cba0 2023-10-16T08:16:34.451 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:16:34.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fced80664a0 msgr2=0x7fced8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fced80664a0 0x7fced8068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcef00af000 tx=0x7fcef0008040 comp rx=0 tx=0).stop 2023-10-16T08:16:34.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 msgr2=0x7fcef414e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef414e140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fceec007ec0 tx=0x7fceec00d4a0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 shutdown_connections 2023-10-16T08:16:34.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcef415cba0 0x7fcef414e140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fced80664a0 0x7fced8068960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcef40d0210 0x7fcef414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 --2- 172.21.15.32:0/2436220614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcef40b84e0 0x7fcef41552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 >> 172.21.15.32:0/2436220614 conn(0x7fcef40719c0 msgr2=0x7fcef406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 shutdown_connections 2023-10-16T08:16:34.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.447+0000 7fcefb26b640 1 -- 172.21.15.32:0/2436220614 wait complete. 2023-10-16T08:16:34.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:16:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcfa60c9640 1 Processor -- start 2023-10-16T08:16:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcfa60c9640 1 -- start start 2023-10-16T08:16:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcfa60c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcfa0070a00 0x7fcfa0163f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcfa60c9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00c9600 con 0x7fcfa006df40 2023-10-16T08:16:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcfa60c9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00c98e0 con 0x7fcfa01590d0 2023-10-16T08:16:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcfa60c9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa00c9bc0 con 0x7fcfa0070a00 2023-10-16T08:16:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcfa0070a00 0x7fcfa0163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf97fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcfa006df40 0x7fcfa00702f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33880/0 (socket says 172.21.15.32:33880) 2023-10-16T08:16:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf97fff640 1 -- 172.21.15.32:0/2688655487 learned_addr learned my addr 172.21.15.32:0/2688655487 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf9ffff640 1 -- 172.21.15.32:0/2688655487 >> v1:172.21.15.186:6789/0 conn(0x7fcfa01590d0 legacy=0x7fcfa00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf9ffff640 1 -- 172.21.15.32:0/2688655487 >> v1:172.21.15.32:6789/0 conn(0x7fcfa006df40 legacy=0x7fcfa00702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf9effd640 1 -- 172.21.15.32:0/2688655487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376701287 0 0) 0x7fcfa00c9600 con 0x7fcfa006df40 2023-10-16T08:16:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.615+0000 7fcf9ffff640 1 -- 172.21.15.32:0/2688655487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00c9ea0 con 0x7fcfa0070a00 2023-10-16T08:16:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcf9ffff640 1 --2- 172.21.15.32:0/2688655487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa0070a00 0x7fcfa0163f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009d90 tx=0x7fcf9801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6877ce6181aaf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcf9effd640 1 -- 172.21.15.32:0/2688655487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980a58a0 con 0x7fcfa0070a00 2023-10-16T08:16:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcf9effd640 1 -- 172.21.15.32:0/2688655487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf980a5a00 con 0x7fcfa0070a00 2023-10-16T08:16:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcf9effd640 1 -- 172.21.15.32:0/2688655487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980a5d90 con 0x7fcfa0070a00 2023-10-16T08:16:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 -- 172.21.15.32:0/2688655487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa0070a00 msgr2=0x7fcfa0163f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/2688655487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa0070a00 0x7fcfa0163f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009d90 tx=0x7fcf9801c950 comp rx=0 tx=0).stop 2023-10-16T08:16:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 -- 172.21.15.32:0/2688655487 shutdown_connections 2023-10-16T08:16:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/2688655487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa0070a00 0x7fcfa0163f80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 -- 172.21.15.32:0/2688655487 >> 172.21.15.32:0/2688655487 conn(0x7fcfa00b0da0 msgr2=0x7fcfa00ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 -- 172.21.15.32:0/2688655487 shutdown_connections 2023-10-16T08:16:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.619+0000 7fcfa60c9640 1 -- 172.21.15.32:0/2688655487 wait complete. 2023-10-16T08:16:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 Processor -- start 2023-10-16T08:16:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- start start 2023-10-16T08:16:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa006df40 0x7fcfa00c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfa0070a00 0x7fcfa00c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 0x7fcfa00c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00c7e50 con 0x7fcfa01590d0 2023-10-16T08:16:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00c22b0 con 0x7fcfa006df40 2023-10-16T08:16:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00c2500 con 0x7fcfa0070a00 2023-10-16T08:16:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 0x7fcfa00c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 0x7fcfa00c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36162/0 (socket says 172.21.15.32:36162) 2023-10-16T08:16:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 -- 172.21.15.32:0/3358146340 learned_addr learned my addr 172.21.15.32:0/3358146340 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf97fff640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfa0070a00 0x7fcfa00c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9f7fe640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa006df40 0x7fcfa00c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 -- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfa0070a00 msgr2=0x7fcfa00c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfa0070a00 0x7fcfa00c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 -- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa006df40 msgr2=0x7fcfa00c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa006df40 0x7fcfa00c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 -- 172.21.15.32:0/3358146340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98009a70 con 0x7fcfa01590d0 2023-10-16T08:16:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf97fff640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfa0070a00 0x7fcfa00c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9ffff640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 0x7fcfa00c41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf980b0040 tx=0x7fcf980196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980b9860 con 0x7fcfa01590d0 2023-10-16T08:16:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf980b99c0 con 0x7fcfa01590d0 2023-10-16T08:16:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980b9d70 con 0x7fcfa01590d0 2023-10-16T08:16:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa00ca820 con 0x7fcfa01590d0 2023-10-16T08:16:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa00c2ac0 con 0x7fcfa01590d0 2023-10-16T08:16:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.623+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa004e170 con 0x7fcfa01590d0 2023-10-16T08:16:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.627+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf980c4460 con 0x7fcfa01590d0 2023-10-16T08:16:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.627+0000 7fcf9cff9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf800664a0 0x7fcf80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.627+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcf9811ffe0 con 0x7fcfa01590d0 2023-10-16T08:16:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.627+0000 7fcf9f7fe640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf800664a0 0x7fcf80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.627+0000 7fcf9f7fe640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf800664a0 0x7fcf80068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf90000f70 tx=0x7fcf90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.631+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf980ed190 con 0x7fcfa01590d0 2023-10-16T08:16:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfa0070700 con 0x7fcfa01590d0 2023-10-16T08:16:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcf9cff9640 1 -- 172.21.15.32:0/3358146340 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf980f07b0 con 0x7fcfa01590d0 2023-10-16T08:16:34.791 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf800664a0 msgr2=0x7fcf80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf800664a0 0x7fcf80068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf90000f70 tx=0x7fcf90009040 comp rx=0 tx=0).stop 2023-10-16T08:16:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 msgr2=0x7fcfa00c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 0x7fcfa00c41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf980b0040 tx=0x7fcf980196f0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 shutdown_connections 2023-10-16T08:16:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfa01590d0 0x7fcfa00c41c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf800664a0 0x7fcf80068960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcfa0070a00 0x7fcfa00c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 --2- 172.21.15.32:0/3358146340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa006df40 0x7fcfa00c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 >> 172.21.15.32:0/3358146340 conn(0x7fcfa00b0da0 msgr2=0x7fcfa00ab920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 shutdown_connections 2023-10-16T08:16:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.787+0000 7fcfa60c9640 1 -- 172.21.15.32:0/3358146340 wait complete. 2023-10-16T08:16:34.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:16:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 Processor -- start 2023-10-16T08:16:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 -- start start 2023-10-16T08:16:34.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819815cba0 0x7f81980d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81980b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980d1670 con 0x7f81980b84e0 2023-10-16T08:16:34.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980d1460 con 0x7f819815cba0 2023-10-16T08:16:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819f53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980d1950 con 0x7f81980d0210 2023-10-16T08:16:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819d2da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f819815cba0 0x7f81980d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819cad9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81980b84e0 0x7f81980cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33888/0 (socket says 172.21.15.32:33888) 2023-10-16T08:16:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819cad9640 1 -- 172.21.15.32:0/1096621077 learned_addr learned my addr 172.21.15.32:0/1096621077 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819dadb640 1 --2- 172.21.15.32:0/1096621077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81980b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819d2da640 1 -- 172.21.15.32:0/1096621077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 msgr2=0x7f81980b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:34.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819d2da640 1 --2- 172.21.15.32:0/1096621077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81980b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819d2da640 1 -- 172.21.15.32:0/1096621077 >> v1:172.21.15.32:6789/0 conn(0x7f81980b84e0 legacy=0x7f81980cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f819d2da640 1 -- 172.21.15.32:0/1096621077 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81980d1c30 con 0x7f819815cba0 2023-10-16T08:16:34.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.954+0000 7f818ffff640 1 -- 172.21.15.32:0/1096621077 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550965818 0 0) 0x7f81980d1670 con 0x7f81980b84e0 2023-10-16T08:16:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819d2da640 1 --2- 172.21.15.32:0/1096621077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f81980d0f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f81940097c0 tx=0x7f819401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8101346de82f8005 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f818ffff640 1 -- 172.21.15.32:0/1096621077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940ab680 con 0x7f819815cba0 2023-10-16T08:16:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f818ffff640 1 -- 172.21.15.32:0/1096621077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81940ab7e0 con 0x7f819815cba0 2023-10-16T08:16:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f818ffff640 1 -- 172.21.15.32:0/1096621077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940abb70 con 0x7f819815cba0 2023-10-16T08:16:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 -- 172.21.15.32:0/1096621077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 msgr2=0x7f81980d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 --2- 172.21.15.32:0/1096621077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f81980d0f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f81940097c0 tx=0x7f819401c480 comp rx=0 tx=0).stop 2023-10-16T08:16:34.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 -- 172.21.15.32:0/1096621077 shutdown_connections 2023-10-16T08:16:34.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 --2- 172.21.15.32:0/1096621077 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81980b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 --2- 172.21.15.32:0/1096621077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f81980d0f20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 -- 172.21.15.32:0/1096621077 >> 172.21.15.32:0/1096621077 conn(0x7f81980719c0 msgr2=0x7f8198071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:34.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 -- 172.21.15.32:0/1096621077 shutdown_connections 2023-10-16T08:16:34.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.958+0000 7f819f53e640 1 -- 172.21.15.32:0/1096621077 wait complete. 2023-10-16T08:16:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 Processor -- start 2023-10-16T08:16:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 -- start start 2023-10-16T08:16:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81980b84e0 0x7f819814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81981561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f8198156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980ca5f0 con 0x7f819815cba0 2023-10-16T08:16:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981f92f0 con 0x7f81980b84e0 2023-10-16T08:16:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980c9cc0 con 0x7f81980d0210 2023-10-16T08:16:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f8198156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f8198156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36174/0 (socket says 172.21.15.32:36174) 2023-10-16T08:16:34.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 -- 172.21.15.32:0/2984100665 learned_addr learned my addr 172.21.15.32:0/2984100665 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:34.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819cad9640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81981561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819d2da640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81980b84e0 0x7f819814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 -- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 msgr2=0x7f81981561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81981561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 -- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81980b84e0 msgr2=0x7f819814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81980b84e0 0x7f819814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 -- 172.21.15.32:0/2984100665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194009420 con 0x7f819815cba0 2023-10-16T08:16:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819dadb640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f8198156710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8190007e30 tx=0x7f819000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190014620 con 0x7f819815cba0 2023-10-16T08:16:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819000ee90 con 0x7f819815cba0 2023-10-16T08:16:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900149c0 con 0x7f819815cba0 2023-10-16T08:16:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81980c9fa0 con 0x7f819815cba0 2023-10-16T08:16:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.962+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81980c6580 con 0x7f819815cba0 2023-10-16T08:16:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.966+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81980ca760 con 0x7f819815cba0 2023-10-16T08:16:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.966+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8190014b20 con 0x7f819815cba0 2023-10-16T08:16:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.966+0000 7f818dffb640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f817c0664a0 0x7f817c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.966+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f81900849e0 con 0x7f819815cba0 2023-10-16T08:16:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.966+0000 7f819d2da640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f817c0664a0 0x7f817c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.966+0000 7f819d2da640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f817c0664a0 0x7f817c068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81940af000 tx=0x7f81940a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:34.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:34.970+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8190051c10 con 0x7f819815cba0 2023-10-16T08:16:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f819804e170 con 0x7f819815cba0 2023-10-16T08:16:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f818dffb640 1 -- 172.21.15.32:0/2984100665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8190055230 con 0x7f819815cba0 2023-10-16T08:16:35.135 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:16:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f817c0664a0 msgr2=0x7f817c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f817c0664a0 0x7f817c068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81940af000 tx=0x7f81940a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 msgr2=0x7f8198156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f8198156710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8190007e30 tx=0x7f819000e4b0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 shutdown_connections 2023-10-16T08:16:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819815cba0 0x7f8198156710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f817c0664a0 0x7f817c068960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81980d0210 0x7f81981561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 --2- 172.21.15.32:0/2984100665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81980b84e0 0x7f819814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 >> 172.21.15.32:0/2984100665 conn(0x7f81980719c0 msgr2=0x7f819815ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 shutdown_connections 2023-10-16T08:16:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.130+0000 7f819f53e640 1 -- 172.21.15.32:0/2984100665 wait complete. 2023-10-16T08:16:35.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:16:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 Processor -- start 2023-10-16T08:16:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 -- start start 2023-10-16T08:16:35.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ffc06a0f0 0x7f4ffc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc06c6d0 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc06c950 con 0x7f4ffc06a0f0 2023-10-16T08:16:35.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f500313e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1654f0 con 0x7f4ffc0ade00 2023-10-16T08:16:35.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f5000eda640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ffc06a0f0 0x7f4ffc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f4ffbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ffc06a4d0 0x7f4ffc0ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33890/0 (socket says 172.21.15.32:33890) 2023-10-16T08:16:35.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f4ffbfff640 1 -- 172.21.15.32:0/1013944982 learned_addr learned my addr 172.21.15.32:0/1013944982 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f5000eda640 1 -- 172.21.15.32:0/1013944982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 msgr2=0x7f4ffc0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f50016db640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f4ffb7fe640 1 -- 172.21.15.32:0/1013944982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883909725 0 0) 0x7f4ffc06c6d0 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f5000eda640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f5000eda640 1 -- 172.21.15.32:0/1013944982 >> v1:172.21.15.32:6789/0 conn(0x7f4ffc06a4d0 legacy=0x7f4ffc0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.298+0000 7f5000eda640 1 -- 172.21.15.32:0/1013944982 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc1657d0 con 0x7f4ffc06a0f0 2023-10-16T08:16:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f50016db640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f5000eda640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c8dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00097c0 tx=0x7f4ff001cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1144552ef57f3c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f4ffb7fe640 1 -- 172.21.15.32:0/1013944982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00a6040 con 0x7f4ffc06a0f0 2023-10-16T08:16:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f4ffb7fe640 1 -- 172.21.15.32:0/1013944982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff00aa600 con 0x7f4ffc06a0f0 2023-10-16T08:16:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f4ffb7fe640 1 -- 172.21.15.32:0/1013944982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00aa990 con 0x7f4ffc06a0f0 2023-10-16T08:16:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f500313e640 1 -- 172.21.15.32:0/1013944982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a0f0 msgr2=0x7f4ffc0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f500313e640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c8dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00097c0 tx=0x7f4ff001cee0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f500313e640 1 -- 172.21.15.32:0/1013944982 shutdown_connections 2023-10-16T08:16:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f500313e640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.302+0000 7f500313e640 1 --2- 172.21.15.32:0/1013944982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c8dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- 172.21.15.32:0/1013944982 >> 172.21.15.32:0/1013944982 conn(0x7f4ffc0b0da0 msgr2=0x7f4ffc0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- 172.21.15.32:0/1013944982 shutdown_connections 2023-10-16T08:16:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- 172.21.15.32:0/1013944982 wait complete. 2023-10-16T08:16:35.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 Processor -- start 2023-10-16T08:16:35.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- start start 2023-10-16T08:16:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 0x7f4ffc0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc15ad90 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0b2800 con 0x7f4ffc06a0f0 2023-10-16T08:16:35.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f500313e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c2da0 con 0x7f4ffc0ade00 2023-10-16T08:16:35.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 0x7f4ffc0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 0x7f4ffc0c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36202/0 (socket says 172.21.15.32:36202) 2023-10-16T08:16:35.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 -- 172.21.15.32:0/3207802482 learned_addr learned my addr 172.21.15.32:0/3207802482 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:35.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f50016db640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f5000eda640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 -- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 msgr2=0x7f4ffc0c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 -- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc06a0f0 msgr2=0x7f4ffc0c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 -- 172.21.15.32:0/3207802482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009420 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f5000eda640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:35.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.306+0000 7f4ffbfff640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 0x7f4ffc0c7b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8007fd0 tx=0x7f4fe800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:35.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8005cf0 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe8005e70 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80072f0 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc0c3080 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc0b2d30 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc04e170 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4fe8007450 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f4ff97fa640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4fe0066520 0x7f4fe00689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4fe80837e0 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.310+0000 7f5000eda640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4fe0066520 0x7f4fe00689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.314+0000 7f5000eda640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4fe0066520 0x7f4fe00689e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00ae000 tx=0x7f4ff0004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:35.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.314+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4fe8050950 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ffc0cc5d0 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f4ff97fa640 1 -- 172.21.15.32:0/3207802482 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4fe8053f70 con 0x7f4ffc06a4d0 2023-10-16T08:16:35.476 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4fe0066520 msgr2=0x7f4fe00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4fe0066520 0x7f4fe00689e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00ae000 tx=0x7f4ff0004740 comp rx=0 tx=0).stop 2023-10-16T08:16:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 msgr2=0x7f4ffc0c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 0x7f4ffc0c7b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8007fd0 tx=0x7f4fe800c8e0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 shutdown_connections 2023-10-16T08:16:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4fe0066520 0x7f4fe00689e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ffc0ade00 0x7f4ffc0c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ffc06a4d0 0x7f4ffc0c7b40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 --2- 172.21.15.32:0/3207802482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ffc06a0f0 0x7f4ffc0c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 >> 172.21.15.32:0/3207802482 conn(0x7f4ffc0b0da0 msgr2=0x7f4ffc0701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 shutdown_connections 2023-10-16T08:16:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.470+0000 7f500313e640 1 -- 172.21.15.32:0/3207802482 wait complete. 2023-10-16T08:16:35.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:16:35.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.634+0000 7fc89ed80640 1 Processor -- start 2023-10-16T08:16:35.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.634+0000 7fc89ed80640 1 -- start start 2023-10-16T08:16:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.634+0000 7fc89ed80640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc89815ca70 0x7fc8980d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89ed80640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980cf7a0 0x7fc8980c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89ed80640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8980c65f0 con 0x7fc8980d3550 2023-10-16T08:16:35.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89ed80640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8980ce820 con 0x7fc89815ca70 2023-10-16T08:16:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89ed80640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8980cdf80 con 0x7fc8980cf7a0 2023-10-16T08:16:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89cb1c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc89815ca70 0x7fc8980d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc897fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc8980d3550 0x7fc8980cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33898/0 (socket says 172.21.15.32:33898) 2023-10-16T08:16:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc897fff640 1 -- 172.21.15.32:0/313537732 learned_addr learned my addr 172.21.15.32:0/313537732 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:35.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89d31d640 1 --2- 172.21.15.32:0/313537732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980cf7a0 0x7fc8980c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89cb1c640 1 -- 172.21.15.32:0/313537732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980cf7a0 msgr2=0x7fc8980c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:35.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89cb1c640 1 --2- 172.21.15.32:0/313537732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980cf7a0 0x7fc8980c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89cb1c640 1 -- 172.21.15.32:0/313537732 >> v1:172.21.15.32:6789/0 conn(0x7fc8980d3550 legacy=0x7fc8980cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89cb1c640 1 -- 172.21.15.32:0/313537732 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8980ce260 con 0x7fc89815ca70 2023-10-16T08:16:35.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc8977fe640 1 -- 172.21.15.32:0/313537732 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091212626 0 0) 0x7fc8980c65f0 con 0x7fc8980d3550 2023-10-16T08:16:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc89cb1c640 1 --2- 172.21.15.32:0/313537732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89815ca70 0x7fc8980d3010 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097a0 tx=0x7fc88c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c959ae8918e40d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:35.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc8977fe640 1 -- 172.21.15.32:0/313537732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0ad640 con 0x7fc89815ca70 2023-10-16T08:16:35.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc8977fe640 1 -- 172.21.15.32:0/313537732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c0ad7a0 con 0x7fc89815ca70 2023-10-16T08:16:35.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.638+0000 7fc8977fe640 1 -- 172.21.15.32:0/313537732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0adb30 con 0x7fc89815ca70 2023-10-16T08:16:35.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- 172.21.15.32:0/313537732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89815ca70 msgr2=0x7fc8980d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 --2- 172.21.15.32:0/313537732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89815ca70 0x7fc8980d3010 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097a0 tx=0x7fc88c01c430 comp rx=0 tx=0).stop 2023-10-16T08:16:35.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- 172.21.15.32:0/313537732 shutdown_connections 2023-10-16T08:16:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 --2- 172.21.15.32:0/313537732 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980cf7a0 0x7fc8980c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 --2- 172.21.15.32:0/313537732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89815ca70 0x7fc8980d3010 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- 172.21.15.32:0/313537732 >> 172.21.15.32:0/313537732 conn(0x7fc89809bc50 msgr2=0x7fc8980d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- 172.21.15.32:0/313537732 shutdown_connections 2023-10-16T08:16:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- 172.21.15.32:0/313537732 wait complete. 2023-10-16T08:16:35.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 Processor -- start 2023-10-16T08:16:35.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- start start 2023-10-16T08:16:35.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 0x7fc89806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980d3550 0x7fc898159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc89815ca70 0x7fc89814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89814f860 con 0x7fc8980cf7a0 2023-10-16T08:16:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89814e2f0 con 0x7fc89815ca70 2023-10-16T08:16:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89ed80640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc898158540 con 0x7fc8980d3550 2023-10-16T08:16:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89cb1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 0x7fc89806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89cb1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 0x7fc89806bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36214/0 (socket says 172.21.15.32:36214) 2023-10-16T08:16:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89cb1c640 1 -- 172.21.15.32:0/3214674224 learned_addr learned my addr 172.21.15.32:0/3214674224 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc89d31d640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc89815ca70 0x7fc89814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.642+0000 7fc897fff640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980d3550 0x7fc898159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89cb1c640 1 -- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980d3550 msgr2=0x7fc898159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89cb1c640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980d3550 0x7fc898159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89cb1c640 1 -- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc89815ca70 msgr2=0x7fc89814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89cb1c640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc89815ca70 0x7fc89814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89cb1c640 1 -- 172.21.15.32:0/3214674224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0093b0 con 0x7fc8980cf7a0 2023-10-16T08:16:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89d31d640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc89815ca70 0x7fc89814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89cb1c640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 0x7fc89806bad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc88c01a060 tx=0x7fc88c0191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0b6610 con 0x7fc8980cf7a0 2023-10-16T08:16:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c0a6040 con 0x7fc8980cf7a0 2023-10-16T08:16:35.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0b69b0 con 0x7fc8980cf7a0 2023-10-16T08:16:35.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc898157f80 con 0x7fc8980cf7a0 2023-10-16T08:16:35.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8981505c0 con 0x7fc8980cf7a0 2023-10-16T08:16:35.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc898074010 con 0x7fc8980cf7a0 2023-10-16T08:16:35.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc88c0b4040 con 0x7fc8980cf7a0 2023-10-16T08:16:35.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.646+0000 7fc8957fa640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc87c0664a0 0x7fc87c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:35.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.650+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc88c11e680 con 0x7fc8980cf7a0 2023-10-16T08:16:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.650+0000 7fc897fff640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc87c0664a0 0x7fc87c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.650+0000 7fc897fff640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc87c0664a0 0x7fc87c068960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8981f87c0 tx=0x7fc884007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.654+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc88c0eb770 con 0x7fc8980cf7a0 2023-10-16T08:16:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.818+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc898065df0 con 0x7fc8980cf7a0 2023-10-16T08:16:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc8957fa640 1 -- 172.21.15.32:0/3214674224 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc88c0b2080 con 0x7fc8980cf7a0 2023-10-16T08:16:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:16:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc87c0664a0 msgr2=0x7fc87c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc87c0664a0 0x7fc87c068960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8981f87c0 tx=0x7fc884007b60 comp rx=0 tx=0).stop 2023-10-16T08:16:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 msgr2=0x7fc89806bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 0x7fc89806bad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc88c01a060 tx=0x7fc88c0191d0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 shutdown_connections 2023-10-16T08:16:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc89815ca70 0x7fc89814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc87c0664a0 0x7fc87c068960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc8980d3550 0x7fc898159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 --2- 172.21.15.32:0/3214674224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8980cf7a0 0x7fc89806bad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 >> 172.21.15.32:0/3214674224 conn(0x7fc89809bc50 msgr2=0x7fc89806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 shutdown_connections 2023-10-16T08:16:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:35.822+0000 7fc89ed80640 1 -- 172.21.15.32:0/3214674224 wait complete. 2023-10-16T08:16:35.847 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:16:35.847 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:16:35.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 Processor -- start 2023-10-16T08:16:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 -- start start 2023-10-16T08:16:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc88c0b00e0 0x7fc88c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 0x7fc88c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c0cb1d0 con 0x7fc88c15a880 2023-10-16T08:16:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c0b0ab0 con 0x7fc88c0b00e0 2023-10-16T08:16:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc8929aa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0cb4b0 con 0x7fc88c0cca20 2023-10-16T08:16:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc88bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc88c15a880 0x7fc88c15dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33906/0 (socket says 172.21.15.32:33906) 2023-10-16T08:16:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc88bfff640 1 -- 172.21.15.32:0/1112224440 learned_addr learned my addr 172.21.15.32:0/1112224440 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc890f47640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 0x7fc88c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.038+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/1112224440 >> v2:172.21.15.186:3300/0 conn(0x7fc88c0b00e0 0x7fc88c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728103330 0 0) 0x7fc88c0cb1d0 con 0x7fc88c15a880 2023-10-16T08:16:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc874003530 con 0x7fc88c15a880 2023-10-16T08:16:36.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2539015146 0 0) 0x7fc874003530 con 0x7fc88c15a880 2023-10-16T08:16:36.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 msgr2=0x7fc88c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:36.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 0x7fc88c0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 >> v2:172.21.15.186:3300/0 conn(0x7fc88c0b00e0 msgr2=0x7fc88c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:36.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 --2- 172.21.15.32:0/1112224440 >> v2:172.21.15.186:3300/0 conn(0x7fc88c0b00e0 0x7fc88c0b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0cb790 con 0x7fc88c15a880 2023-10-16T08:16:36.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fc87c0016a0 con 0x7fc88c15a880 2023-10-16T08:16:36.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 >> v1:172.21.15.32:6789/0 conn(0x7fc88c15a880 legacy=0x7fc88c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 0x7fc874003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc8740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc874004d10 0x7fc874005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b0ab0 con 0x7fc874003f00 2023-10-16T08:16:36.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0cb4b0 con 0x7fc874004d10 2023-10-16T08:16:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87400b990 con 0x7fc88c0cca20 2023-10-16T08:16:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc8740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc890f47640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc874004d10 0x7fc874005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88bfff640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 0x7fc874003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88b7fe640 1 -- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 msgr2=0x7fc874003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0cca20 0x7fc874003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88b7fe640 1 -- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc874004d10 msgr2=0x7fc874005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc874004d10 0x7fc874005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc88b7fe640 1 -- 172.21.15.32:0/1112224440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874003530 con 0x7fc874003f00 2023-10-16T08:16:36.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.042+0000 7fc890f47640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc874004d10 0x7fc874005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:36.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc8740047d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ae0 tx=0x7fc88001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e38b4f901007e19 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8800a5570 con 0x7fc874003f00 2023-10-16T08:16:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8800a56d0 con 0x7fc874003f00 2023-10-16T08:16:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc88affd640 1 -- 172.21.15.32:0/1112224440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8800a5a80 con 0x7fc874003f00 2023-10-16T08:16:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- 172.21.15.32:0/1112224440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 msgr2=0x7fc8740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 --2- 172.21.15.32:0/1112224440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc8740047d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ae0 tx=0x7fc88001ca30 comp rx=0 tx=0).stop 2023-10-16T08:16:36.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc890f47640 1 -- 172.21.15.32:0/1112224440 reap_dead start 2023-10-16T08:16:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- 172.21.15.32:0/1112224440 shutdown_connections 2023-10-16T08:16:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- 172.21.15.32:0/1112224440 >> 172.21.15.32:0/1112224440 conn(0x7fc88c06c190 msgr2=0x7fc88c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- 172.21.15.32:0/1112224440 shutdown_connections 2023-10-16T08:16:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- 172.21.15.32:0/1112224440 wait complete. 2023-10-16T08:16:36.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 Processor -- start 2023-10-16T08:16:36.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- start start 2023-10-16T08:16:36.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc88c0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc88c0b3f50 0x7fc88c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0b0ed0 0x7fc88c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b1720 con 0x7fc874003f00 2023-10-16T08:16:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b2650 con 0x7fc88c0b3f50 2023-10-16T08:16:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.046+0000 7fc8929aa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c15a270 con 0x7fc88c0b0ed0 2023-10-16T08:16:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc88c0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc88c0c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36228/0 (socket says 172.21.15.32:36228) 2023-10-16T08:16:36.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 -- 172.21.15.32:0/155852427 learned_addr learned my addr 172.21.15.32:0/155852427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc88c0b3f50 0x7fc88c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc890f47640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0b0ed0 0x7fc88c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 -- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0b0ed0 msgr2=0x7fc88c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0b0ed0 0x7fc88c15bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 -- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc88c0b3f50 msgr2=0x7fc88c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc88c0b3f50 0x7fc88c159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 -- 172.21.15.32:0/155852427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0c6b60 con 0x7fc874003f00 2023-10-16T08:16:36.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc88bfff640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc88c0c1f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc87c011950 tx=0x7fc87c011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00fc20 con 0x7fc874003f00 2023-10-16T08:16:36.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c00fd80 con 0x7fc874003f00 2023-10-16T08:16:36.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0c6e40 con 0x7fc874003f00 2023-10-16T08:16:36.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c015630 con 0x7fc874003f00 2023-10-16T08:16:36.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c0c7400 con 0x7fc874003f00 2023-10-16T08:16:36.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c0c7620 con 0x7fc874003f00 2023-10-16T08:16:36.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.050+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc87c015790 con 0x7fc874003f00 2023-10-16T08:16:36.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.054+0000 7fc888ff9640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc8780664d0 0x7fc878068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.054+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc87c0874d0 con 0x7fc874003f00 2023-10-16T08:16:36.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.054+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc8780664d0 0x7fc878068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.054+0000 7fc88b7fe640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc8780664d0 0x7fc878068990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8800af000 tx=0x7fc8800046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.054+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc87c054670 con 0x7fc874003f00 2023-10-16T08:16:36.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.214+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc88c0c9070 con 0x7fc874003f00 2023-10-16T08:16:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc888ff9640 1 -- 172.21.15.32:0/155852427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fc87c057c90 con 0x7fc874003f00 2023-10-16T08:16:36.223 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:36.223 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc8780664d0 msgr2=0x7fc878068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc8780664d0 0x7fc878068990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8800af000 tx=0x7fc8800046d0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 msgr2=0x7fc88c0c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc88c0c1f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc87c011950 tx=0x7fc87c011e10 comp rx=0 tx=0).stop 2023-10-16T08:16:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 shutdown_connections 2023-10-16T08:16:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc8780664d0 0x7fc878068990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc88c0b0ed0 0x7fc88c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc88c0b3f50 0x7fc88c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 --2- 172.21.15.32:0/155852427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc874003f00 0x7fc88c0c1f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 >> 172.21.15.32:0/155852427 conn(0x7fc88c06c190 msgr2=0x7fc88c0710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 shutdown_connections 2023-10-16T08:16:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.218+0000 7fc8929aa640 1 -- 172.21.15.32:0/155852427 wait complete. 2023-10-16T08:16:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:16:36.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:16:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 Processor -- start 2023-10-16T08:16:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 -- start start 2023-10-16T08:16:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d80d0210 0x7f66d80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f66d80b84e0 0x7f66d80b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80d1670 con 0x7f66d815cba0 2023-10-16T08:16:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80d1460 con 0x7f66d80b84e0 2023-10-16T08:16:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66dd3f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80d1950 con 0x7f66d80d0210 2023-10-16T08:16:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f66d80b84e0 0x7f66d80b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66ce7fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d80d0210 0x7f66d80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f66d80b84e0 0x7f66d80b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:43360/0 (socket says 172.21.15.32:43360) 2023-10-16T08:16:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 -- 172.21.15.32:0/517243774 learned_addr learned my addr 172.21.15.32:0/517243774 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 -- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d80d0210 msgr2=0x7f66d80d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 --2- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d80d0210 0x7f66d80d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 -- 172.21.15.32:0/517243774 >> v1:172.21.15.186:6789/0 conn(0x7f66d815cba0 legacy=0x7f66d80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d77fe640 1 -- 172.21.15.32:0/517243774 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80d1c30 con 0x7f66d80b84e0 2023-10-16T08:16:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.386+0000 7f66d67fc640 1 -- 172.21.15.32:0/517243774 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656929029 0 0) 0x7f66d80d1670 con 0x7f66d815cba0 2023-10-16T08:16:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66d77fe640 1 --2- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d80b5cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009e50 tx=0x7f66d001ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97074af06377217b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66d67fc640 1 -- 172.21.15.32:0/517243774 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00a58e0 con 0x7f66d80b84e0 2023-10-16T08:16:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66d67fc640 1 -- 172.21.15.32:0/517243774 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d00a5a40 con 0x7f66d80b84e0 2023-10-16T08:16:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66d67fc640 1 -- 172.21.15.32:0/517243774 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00a5dd0 con 0x7f66d80b84e0 2023-10-16T08:16:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 -- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 msgr2=0x7f66d80b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d80b5cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009e50 tx=0x7f66d001ca10 comp rx=0 tx=0).stop 2023-10-16T08:16:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 -- 172.21.15.32:0/517243774 shutdown_connections 2023-10-16T08:16:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d80d0210 0x7f66d80d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/517243774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d80b5cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 -- 172.21.15.32:0/517243774 >> 172.21.15.32:0/517243774 conn(0x7f66d80719c0 msgr2=0x7f66d8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 -- 172.21.15.32:0/517243774 shutdown_connections 2023-10-16T08:16:36.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.390+0000 7f66dd3f8640 1 -- 172.21.15.32:0/517243774 wait complete. 2023-10-16T08:16:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 Processor -- start 2023-10-16T08:16:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 -- start start 2023-10-16T08:16:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d81568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 0x7f66d80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d815cba0 0x7f66d80caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8154480 con 0x7f66d80d0210 2023-10-16T08:16:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8153ec0 con 0x7f66d80b84e0 2023-10-16T08:16:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81541a0 con 0x7f66d815cba0 2023-10-16T08:16:36.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 0x7f66d80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 0x7f66d80b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36238/0 (socket says 172.21.15.32:36238) 2023-10-16T08:16:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 -- 172.21.15.32:0/2089529553 learned_addr learned my addr 172.21.15.32:0/2089529553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66d77fe640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d815cba0 0x7f66d80caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66d6ffd640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d81568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 -- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d815cba0 msgr2=0x7f66d80caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d815cba0 0x7f66d80caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 -- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 msgr2=0x7f66d81568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d81568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 -- 172.21.15.32:0/2089529553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0009b30 con 0x7f66d80d0210 2023-10-16T08:16:36.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66d6ffd640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d81568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66ce7fc640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 0x7f66d80b6c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e930 tx=0x7f66c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0009690 con 0x7f66d80d0210 2023-10-16T08:16:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c000ca00 con 0x7f66d80d0210 2023-10-16T08:16:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c000cd70 con 0x7f66d80d0210 2023-10-16T08:16:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d80c7e90 con 0x7f66d80d0210 2023-10-16T08:16:36.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.394+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d80b7150 con 0x7f66d80d0210 2023-10-16T08:16:36.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.398+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d80b7430 con 0x7f66d80d0210 2023-10-16T08:16:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.398+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f66c0013470 con 0x7f66d80d0210 2023-10-16T08:16:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.398+0000 7f66cffff640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66b80664a0 0x7f66b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.398+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66c0084350 con 0x7f66d80d0210 2023-10-16T08:16:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.398+0000 7f66d6ffd640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66b80664a0 0x7f66b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.398+0000 7f66d6ffd640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66b80664a0 0x7f66b8068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000f70 tx=0x7f66c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.402+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f66d80b7430 con 0x7f66d80d0210 2023-10-16T08:16:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.558+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66d80b7430 con 0x7f66d80d0210 2023-10-16T08:16:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.558+0000 7f66cffff640 1 -- 172.21.15.32:0/2089529553 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66d80b7430 con 0x7f66d80d0210 2023-10-16T08:16:36.566 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:16:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66b80664a0 msgr2=0x7f66b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66b80664a0 0x7f66b8068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000f70 tx=0x7f66c8009040 comp rx=0 tx=0).stop 2023-10-16T08:16:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 msgr2=0x7f66d80b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 0x7f66d80b6c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e930 tx=0x7f66c000edf0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 shutdown_connections 2023-10-16T08:16:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66b80664a0 0x7f66b8068960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66d815cba0 0x7f66d80caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d80d0210 0x7f66d80b6c10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 --2- 172.21.15.32:0/2089529553 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66d80b84e0 0x7f66d81568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 >> 172.21.15.32:0/2089529553 conn(0x7f66d80719c0 msgr2=0x7f66d80cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 shutdown_connections 2023-10-16T08:16:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.562+0000 7f66dd3f8640 1 -- 172.21.15.32:0/2089529553 wait complete. 2023-10-16T08:16:36.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:16:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.726+0000 7f42b9d1d640 1 Processor -- start 2023-10-16T08:16:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.726+0000 7f42b9d1d640 1 -- start start 2023-10-16T08:16:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b9d1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b40d0210 0x7f42b40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b9d1d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42b40b84e0 0x7f42b40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b9d1d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b40d1670 con 0x7f42b415cba0 2023-10-16T08:16:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b9d1d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b40d1460 con 0x7f42b40b84e0 2023-10-16T08:16:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b9d1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40d1950 con 0x7f42b40d0210 2023-10-16T08:16:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42b415cba0 0x7f42b40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33912/0 (socket says 172.21.15.32:33912) 2023-10-16T08:16:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b37fe640 1 -- 172.21.15.32:0/3417242489 learned_addr learned my addr 172.21.15.32:0/3417242489 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b3fff640 1 --2- 172.21.15.32:0/3417242489 >> v2:172.21.15.32:3300/0 conn(0x7f42b40b84e0 0x7f42b40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b2ffd640 1 --2- 172.21.15.32:0/3417242489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b40d0210 0x7f42b40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b3fff640 1 -- 172.21.15.32:0/3417242489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b40d0210 msgr2=0x7f42b40d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b3fff640 1 --2- 172.21.15.32:0/3417242489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b40d0210 0x7f42b40d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b3fff640 1 -- 172.21.15.32:0/3417242489 >> v1:172.21.15.32:6789/0 conn(0x7f42b415cba0 legacy=0x7f42b40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.730+0000 7f42b3fff640 1 -- 172.21.15.32:0/3417242489 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40d1c30 con 0x7f42b40b84e0 2023-10-16T08:16:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b3fff640 1 --2- 172.21.15.32:0/3417242489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b5cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009d90 tx=0x7f42ac01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda200593704b614 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b27fc640 1 -- 172.21.15.32:0/3417242489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0a5a10 con 0x7f42b40b84e0 2023-10-16T08:16:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b27fc640 1 -- 172.21.15.32:0/3417242489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac0a5b70 con 0x7f42b40b84e0 2023-10-16T08:16:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b27fc640 1 -- 172.21.15.32:0/3417242489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0ad440 con 0x7f42b40b84e0 2023-10-16T08:16:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3417242489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 msgr2=0x7f42b40b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3417242489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b5cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009d90 tx=0x7f42ac01c950 comp rx=0 tx=0).stop 2023-10-16T08:16:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3417242489 shutdown_connections 2023-10-16T08:16:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3417242489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b5cf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3417242489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b40d0210 0x7f42b40d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3417242489 >> 172.21.15.32:0/3417242489 conn(0x7f42b40719c0 msgr2=0x7f42b4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3417242489 shutdown_connections 2023-10-16T08:16:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3417242489 wait complete. 2023-10-16T08:16:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 Processor -- start 2023-10-16T08:16:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.734+0000 7f42b9d1d640 1 -- start start 2023-10-16T08:16:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42b40d0210 0x7f42b4150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b415cba0 0x7f42b40c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40b2220 con 0x7f42b40b84e0 2023-10-16T08:16:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4151b40 con 0x7f42b40d0210 2023-10-16T08:16:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4150780 con 0x7f42b415cba0 2023-10-16T08:16:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36260/0 (socket says 172.21.15.32:36260) 2023-10-16T08:16:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 -- 172.21.15.32:0/3452603407 learned_addr learned my addr 172.21.15.32:0/3452603407 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b2ffd640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42b40d0210 0x7f42b4150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b3fff640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b415cba0 0x7f42b40c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 -- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b415cba0 msgr2=0x7f42b40c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b415cba0 0x7f42b40c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 -- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42b40d0210 msgr2=0x7f42b4150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42b40d0210 0x7f42b4150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 -- 172.21.15.32:0/3452603407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac009a70 con 0x7f42b40b84e0 2023-10-16T08:16:36.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b37fe640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b3330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c430 tx=0x7f42a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4005cf0 con 0x7f42b40b84e0 2023-10-16T08:16:36.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4005e50 con 0x7f42b40b84e0 2023-10-16T08:16:36.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40072d0 con 0x7f42b40b84e0 2023-10-16T08:16:36.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b415bfc0 con 0x7f42b40b84e0 2023-10-16T08:16:36.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b4151d90 con 0x7f42b40b84e0 2023-10-16T08:16:36.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b404e170 con 0x7f42b40b84e0 2023-10-16T08:16:36.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.738+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f42a4007430 con 0x7f42b40b84e0 2023-10-16T08:16:36.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.742+0000 7f429bfff640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f42a00664a0 0x7f42a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:36.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.742+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f42a4083560 con 0x7f42b40b84e0 2023-10-16T08:16:36.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.742+0000 7f42b2ffd640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f42a00664a0 0x7f42a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:36.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.742+0000 7f42b2ffd640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f42a00664a0 0x7f42a0068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b41fc880 tx=0x7f42a8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:36.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.746+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f42a4050830 con 0x7f42b40b84e0 2023-10-16T08:16:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.898+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42b4071da0 con 0x7f42b40b84e0 2023-10-16T08:16:36.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.898+0000 7f429bfff640 1 -- 172.21.15.32:0/3452603407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42a4053e50 con 0x7f42b40b84e0 2023-10-16T08:16:36.906 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f42a00664a0 msgr2=0x7f42a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f42a00664a0 0x7f42a0068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b41fc880 tx=0x7f42a8009190 comp rx=0 tx=0).stop 2023-10-16T08:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 msgr2=0x7f42b40b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b3330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c430 tx=0x7f42a400c8f0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 shutdown_connections 2023-10-16T08:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f42a00664a0 0x7f42a0068960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42b415cba0 0x7f42b40c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42b40d0210 0x7f42b4150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 --2- 172.21.15.32:0/3452603407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b40b84e0 0x7f42b40b3330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 >> 172.21.15.32:0/3452603407 conn(0x7f42b40719c0 msgr2=0x7f42b4154690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 shutdown_connections 2023-10-16T08:16:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:36.902+0000 7f42b9d1d640 1 -- 172.21.15.32:0/3452603407 wait complete. 2023-10-16T08:16:36.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:16:37.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.066+0000 7fc7c11d9640 1 Processor -- start 2023-10-16T08:16:37.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.066+0000 7fc7c11d9640 1 -- start start 2023-10-16T08:16:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7c11d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc7bc0b84e0 0x7fc7bc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7c11d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc0d1670 con 0x7fc7bc0d0210 2023-10-16T08:16:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7c11d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc0d1950 con 0x7fc7bc15cba0 2023-10-16T08:16:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7c11d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7bc0d1460 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7b259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7bc0d0210 0x7fc7bc0b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33920/0 (socket says 172.21.15.32:33920) 2023-10-16T08:16:37.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7b259c640 1 -- 172.21.15.32:0/941464845 learned_addr learned my addr 172.21.15.32:0/941464845 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7bb59e640 1 --2- 172.21.15.32:0/941464845 >> v2:172.21.15.186:3300/0 conn(0x7fc7bc0b84e0 0x7fc7bc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7ba59c640 1 -- 172.21.15.32:0/941464845 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365383523 0 0) 0x7fc7bc0d1670 con 0x7fc7bc0d0210 2023-10-16T08:16:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7ba59c640 1 -- 172.21.15.32:0/941464845 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a8003530 con 0x7fc7bc0d0210 2023-10-16T08:16:37.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7bb59e640 1 -- 172.21.15.32:0/941464845 >> v1:172.21.15.186:6789/0 conn(0x7fc7bc15cba0 legacy=0x7fc7bc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7bb59e640 1 -- 172.21.15.32:0/941464845 >> v1:172.21.15.32:6789/0 conn(0x7fc7bc0d0210 legacy=0x7fc7bc0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7ba59c640 1 -- 172.21.15.32:0/941464845 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2862568139 0 0) 0x7fc7a8003530 con 0x7fc7bc0d0210 2023-10-16T08:16:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.070+0000 7fc7bb59e640 1 -- 172.21.15.32:0/941464845 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0d1c30 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7bb59e640 1 --2- 172.21.15.32:0/941464845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0d0f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40097c0 tx=0x7fc7b401c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b95e7702f19bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7ba59c640 1 -- 172.21.15.32:0/941464845 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4001680 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7ba59c640 1 -- 172.21.15.32:0/941464845 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b40a7050 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7ba59c640 1 -- 172.21.15.32:0/941464845 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40ab620 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 -- 172.21.15.32:0/941464845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc0b84e0 msgr2=0x7fc7bc0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/941464845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0d0f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40097c0 tx=0x7fc7b401c310 comp rx=0 tx=0).stop 2023-10-16T08:16:37.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 -- 172.21.15.32:0/941464845 shutdown_connections 2023-10-16T08:16:37.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/941464845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0d0f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 -- 172.21.15.32:0/941464845 >> 172.21.15.32:0/941464845 conn(0x7fc7bc0719c0 msgr2=0x7fc7bc071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:37.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 -- 172.21.15.32:0/941464845 shutdown_connections 2023-10-16T08:16:37.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.074+0000 7fc7c11d9640 1 -- 172.21.15.32:0/941464845 wait complete. 2023-10-16T08:16:37.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 Processor -- start 2023-10-16T08:16:37.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 -- start start 2023-10-16T08:16:37.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7bc0d0210 0x7fc7bc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc15cba0 0x7fc7bc150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc14b3a0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc14a760 con 0x7fc7bc15cba0 2023-10-16T08:16:37.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc14aa00 con 0x7fc7bc0d0210 2023-10-16T08:16:37.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36272/0 (socket says 172.21.15.32:36272) 2023-10-16T08:16:37.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 -- 172.21.15.32:0/2116278133 learned_addr learned my addr 172.21.15.32:0/2116278133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:37.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bb59e640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc15cba0 0x7fc7bc150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7b259c640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7bc0d0210 0x7fc7bc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 -- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7bc0d0210 msgr2=0x7fc7bc159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7bc0d0210 0x7fc7bc159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 -- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc15cba0 msgr2=0x7fc7bc150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc15cba0 0x7fc7bc150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 -- 172.21.15.32:0/2116278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4009420 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7b259c640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7bc0d0210 0x7fc7bc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:37.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7bad9d640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0c52e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c7a0 tx=0x7fc7ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac003b80 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac003ce0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00aa90 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc14ace0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.078+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc159540 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.082+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc04e170 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.082+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc7ac00abf0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.082+0000 7fc7b3fff640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc79c0664a0 0x7fc79c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.082+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc7ac083fb0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.082+0000 7fc7b259c640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc79c0664a0 0x7fc79c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.082+0000 7fc7b259c640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc79c0664a0 0x7fc79c068960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0c5f30 tx=0x7fc7a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.086+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc7ac0510a0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.242+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7bc0720e0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.242+0000 7fc7b3fff640 1 -- 172.21.15.32:0/2116278133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7ac0546c0 con 0x7fc7bc0b84e0 2023-10-16T08:16:37.249 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:16:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc79c0664a0 msgr2=0x7fc79c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc79c0664a0 0x7fc79c068960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0c5f30 tx=0x7fc7a4009040 comp rx=0 tx=0).stop 2023-10-16T08:16:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 msgr2=0x7fc7bc0c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0c52e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c7a0 tx=0x7fc7ac00cc60 comp rx=0 tx=0).stop 2023-10-16T08:16:37.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 shutdown_connections 2023-10-16T08:16:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7bc15cba0 0x7fc7bc150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc79c0664a0 0x7fc79c068960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc7bc0d0210 0x7fc7bc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 --2- 172.21.15.32:0/2116278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7bc0b84e0 0x7fc7bc0c52e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 >> 172.21.15.32:0/2116278133 conn(0x7fc7bc0719c0 msgr2=0x7fc7bc0d2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 shutdown_connections 2023-10-16T08:16:37.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.246+0000 7fc7c11d9640 1 -- 172.21.15.32:0/2116278133 wait complete. 2023-10-16T08:16:37.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:16:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 Processor -- start 2023-10-16T08:16:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 -- start start 2023-10-16T08:16:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34d415feb0 0x7f34d40af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34d40d0830 0x7f34d40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d40d1150 con 0x7f34d40b14c0 2023-10-16T08:16:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d40afc40 con 0x7f34d415feb0 2023-10-16T08:16:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d990f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d40aff20 con 0x7f34d40d0830 2023-10-16T08:16:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34d2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34d415feb0 0x7f34d40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34cbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34d40b14c0 0x7f34d40b0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33922/0 (socket says 172.21.15.32:33922) 2023-10-16T08:16:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.410+0000 7f34cbfff640 1 -- 172.21.15.32:0/4080005135 learned_addr learned my addr 172.21.15.32:0/4080005135 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d37fe640 1 --2- 172.21.15.32:0/4080005135 >> v2:172.21.15.186:3300/0 conn(0x7f34d40d0830 0x7f34d40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d27fc640 1 -- 172.21.15.32:0/4080005135 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100928209 0 0) 0x7f34d40d1150 con 0x7f34d40b14c0 2023-10-16T08:16:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d2ffd640 1 -- 172.21.15.32:0/4080005135 >> v2:172.21.15.186:3300/0 conn(0x7f34d40d0830 msgr2=0x7f34d40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d2ffd640 1 --2- 172.21.15.32:0/4080005135 >> v2:172.21.15.186:3300/0 conn(0x7f34d40d0830 0x7f34d40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d2ffd640 1 -- 172.21.15.32:0/4080005135 >> v1:172.21.15.32:6789/0 conn(0x7f34d40b14c0 legacy=0x7f34d40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d2ffd640 1 -- 172.21.15.32:0/4080005135 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d40b0200 con 0x7f34d415feb0 2023-10-16T08:16:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d37fe640 1 --2- 172.21.15.32:0/4080005135 >> v2:172.21.15.186:3300/0 conn(0x7f34d40d0830 0x7f34d40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d2ffd640 1 --2- 172.21.15.32:0/4080005135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d415feb0 0x7f34d40af410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34d40c9790 tx=0x7f34c401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52c9a4891db6fab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d27fc640 1 -- 172.21.15.32:0/4080005135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c40ad680 con 0x7f34d415feb0 2023-10-16T08:16:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d27fc640 1 -- 172.21.15.32:0/4080005135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c40ad7e0 con 0x7f34d415feb0 2023-10-16T08:16:37.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d27fc640 1 -- 172.21.15.32:0/4080005135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c40adb70 con 0x7f34d415feb0 2023-10-16T08:16:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d990f640 1 -- 172.21.15.32:0/4080005135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d415feb0 msgr2=0x7f34d40af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.414+0000 7f34d990f640 1 --2- 172.21.15.32:0/4080005135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d415feb0 0x7f34d40af410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34d40c9790 tx=0x7f34c401c380 comp rx=0 tx=0).stop 2023-10-16T08:16:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- 172.21.15.32:0/4080005135 shutdown_connections 2023-10-16T08:16:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 --2- 172.21.15.32:0/4080005135 >> v2:172.21.15.186:3300/0 conn(0x7f34d40d0830 0x7f34d40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 --2- 172.21.15.32:0/4080005135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d415feb0 0x7f34d40af410 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- 172.21.15.32:0/4080005135 >> 172.21.15.32:0/4080005135 conn(0x7f34d4089f80 msgr2=0x7f34d40d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- 172.21.15.32:0/4080005135 shutdown_connections 2023-10-16T08:16:37.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- 172.21.15.32:0/4080005135 wait complete. 2023-10-16T08:16:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 Processor -- start 2023-10-16T08:16:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- start start 2023-10-16T08:16:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 0x7f34d40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40d0830 0x7f34d40c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415feb0 0x7f34d40b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40c8b90 con 0x7f34d40b14c0 2023-10-16T08:16:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4159510 con 0x7f34d40d0830 2023-10-16T08:16:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40b7ad0 con 0x7f34d415feb0 2023-10-16T08:16:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 0x7f34d40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 0x7f34d40b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36286/0 (socket says 172.21.15.32:36286) 2023-10-16T08:16:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 -- 172.21.15.32:0/1120559489 learned_addr learned my addr 172.21.15.32:0/1120559489 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d37fe640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415feb0 0x7f34d40b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34cbfff640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40d0830 0x7f34d40c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 -- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415feb0 msgr2=0x7f34d40b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415feb0 0x7f34d40b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 -- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40d0830 msgr2=0x7f34d40c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40d0830 0x7f34d40c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 -- 172.21.15.32:0/1120559489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40093b0 con 0x7f34d40b14c0 2023-10-16T08:16:37.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d2ffd640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 0x7f34d40b3b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f34c4018f90 tx=0x7f34c401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c40ade20 con 0x7f34d40b14c0 2023-10-16T08:16:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c40b8450 con 0x7f34d40b14c0 2023-10-16T08:16:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c40b8800 con 0x7f34d40b14c0 2023-10-16T08:16:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.418+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d40b7db0 con 0x7f34d40b14c0 2023-10-16T08:16:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.422+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d40b8370 con 0x7f34d40b14c0 2023-10-16T08:16:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.422+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d4073fa0 con 0x7f34d40b14c0 2023-10-16T08:16:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.422+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34c40b4070 con 0x7f34d40b14c0 2023-10-16T08:16:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.422+0000 7f34d0ff9640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34b40664a0 0x7f34b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.422+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f34c411e870 con 0x7f34d40b14c0 2023-10-16T08:16:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.422+0000 7f34cbfff640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34b40664a0 0x7f34b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.426+0000 7f34cbfff640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34b40664a0 0x7f34b4068960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34d4153b00 tx=0x7f34bc007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.426+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34c40eb9f0 con 0x7f34d40b14c0 2023-10-16T08:16:37.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.582+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f34d406ad40 con 0x7f34d40b14c0 2023-10-16T08:16:37.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.582+0000 7f34d0ff9640 1 -- 172.21.15.32:0/1120559489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f34c40ef010 con 0x7f34d40b14c0 2023-10-16T08:16:37.590 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:37.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34b40664a0 msgr2=0x7f34b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34b40664a0 0x7f34b4068960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34d4153b00 tx=0x7f34bc007b60 comp rx=0 tx=0).stop 2023-10-16T08:16:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 msgr2=0x7f34d40b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 0x7f34d40b3b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f34c4018f90 tx=0x7f34c401abc0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 shutdown_connections 2023-10-16T08:16:37.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34b40664a0 0x7f34b4068960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34d415feb0 0x7f34d40b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34d40d0830 0x7f34d40c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 --2- 172.21.15.32:0/1120559489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34d40b14c0 0x7f34d40b3b90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 >> 172.21.15.32:0/1120559489 conn(0x7f34d4089f80 msgr2=0x7f34d40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:37.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 shutdown_connections 2023-10-16T08:16:37.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.586+0000 7f34d990f640 1 -- 172.21.15.32:0/1120559489 wait complete. 2023-10-16T08:16:37.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:16:37.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.750+0000 7f6f92ed6640 1 Processor -- start 2023-10-16T08:16:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.750+0000 7f6f92ed6640 1 -- start start 2023-10-16T08:16:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.750+0000 7f6f92ed6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f8c0d3010 0x7f6f8c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.750+0000 7f6f92ed6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c0ce770 con 0x7f6f8c0d15d0 2023-10-16T08:16:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.750+0000 7f6f92ed6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8c0d0600 con 0x7f6f8c15cb90 2023-10-16T08:16:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.750+0000 7f6f92ed6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8c0c6450 con 0x7f6f8c0d3010 2023-10-16T08:16:37.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f8c0d15d0 0x7f6f8c0c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33932/0 (socket says 172.21.15.32:33932) 2023-10-16T08:16:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8bfff640 1 -- 172.21.15.32:0/4152492699 learned_addr learned my addr 172.21.15.32:0/4152492699 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f91473640 1 --2- 172.21.15.32:0/4152492699 >> v2:172.21.15.32:3300/0 conn(0x7f6f8c0d3010 0x7f6f8c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8b7fe640 1 -- 172.21.15.32:0/4152492699 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599080858 0 0) 0x7f6f8c0ce770 con 0x7f6f8c0d15d0 2023-10-16T08:16:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8b7fe640 1 -- 172.21.15.32:0/4152492699 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f7c003530 con 0x7f6f8c0d15d0 2023-10-16T08:16:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f91473640 1 -- 172.21.15.32:0/4152492699 >> v1:172.21.15.186:6789/0 conn(0x7f6f8c15cb90 legacy=0x7f6f8c0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f91473640 1 -- 172.21.15.32:0/4152492699 >> v1:172.21.15.32:6789/0 conn(0x7f6f8c0d15d0 legacy=0x7f6f8c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f91473640 1 -- 172.21.15.32:0/4152492699 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0d08a0 con 0x7f6f8c0d3010 2023-10-16T08:16:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f91473640 1 --2- 172.21.15.32:0/4152492699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c0d3010 0x7f6f8c0cfb10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f8401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4c5eb2a0895a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8b7fe640 1 -- 172.21.15.32:0/4152492699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840a5950 con 0x7f6f8c0d3010 2023-10-16T08:16:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8b7fe640 1 -- 172.21.15.32:0/4152492699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f840a5ab0 con 0x7f6f8c0d3010 2023-10-16T08:16:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.754+0000 7f6f8b7fe640 1 -- 172.21.15.32:0/4152492699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840a5e40 con 0x7f6f8c0d3010 2023-10-16T08:16:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- 172.21.15.32:0/4152492699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c0d3010 msgr2=0x7f6f8c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/4152492699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c0d3010 0x7f6f8c0cfb10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f8401c890 comp rx=0 tx=0).stop 2023-10-16T08:16:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- 172.21.15.32:0/4152492699 shutdown_connections 2023-10-16T08:16:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/4152492699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c0d3010 0x7f6f8c0cfb10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- 172.21.15.32:0/4152492699 >> 172.21.15.32:0/4152492699 conn(0x7f6f8c09bc50 msgr2=0x7f6f8c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- 172.21.15.32:0/4152492699 shutdown_connections 2023-10-16T08:16:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- 172.21.15.32:0/4152492699 wait complete. 2023-10-16T08:16:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 Processor -- start 2023-10-16T08:16:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- start start 2023-10-16T08:16:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f8c0d15d0 0x7f6f8c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8c0d3010 0x7f6f8c06b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 0x7f6f8c159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c0c99d0 con 0x7f6f8c15cb90 2023-10-16T08:16:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c0c9cb0 con 0x7f6f8c0d15d0 2023-10-16T08:16:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f92ed6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c15b630 con 0x7f6f8c0d3010 2023-10-16T08:16:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 0x7f6f8c159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 0x7f6f8c159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36298/0 (socket says 172.21.15.32:36298) 2023-10-16T08:16:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 -- 172.21.15.32:0/1493737138 learned_addr learned my addr 172.21.15.32:0/1493737138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f8bfff640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8c0d3010 0x7f6f8c06b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f90c72640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f8c0d15d0 0x7f6f8c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 -- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8c0d3010 msgr2=0x7f6f8c06b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8c0d3010 0x7f6f8c06b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 -- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f8c0d15d0 msgr2=0x7f6f8c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f8c0d15d0 0x7f6f8c15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f91473640 1 -- 172.21.15.32:0/1493737138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84009a70 con 0x7f6f8c15cb90 2023-10-16T08:16:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.758+0000 7f6f8bfff640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8c0d3010 0x7f6f8c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f91473640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 0x7f6f8c159780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f84019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840add90 con 0x7f6f8c15cb90 2023-10-16T08:16:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f84009450 con 0x7f6f8c15cb90 2023-10-16T08:16:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c0b2e20 con 0x7f6f8c15cb90 2023-10-16T08:16:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840b9690 con 0x7f6f8c15cb90 2023-10-16T08:16:37.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c0b33e0 con 0x7f6f8c15cb90 2023-10-16T08:16:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c074010 con 0x7f6f8c15cb90 2023-10-16T08:16:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6f840b97f0 con 0x7f6f8c15cb90 2023-10-16T08:16:37.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f897fa640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f700664a0 0x7f6f70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:37.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6f8411f9c0 con 0x7f6f8c15cb90 2023-10-16T08:16:37.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.762+0000 7f6f90c72640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f700664a0 0x7f6f70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:37.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.766+0000 7f6f90c72640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f700664a0 0x7f6f70068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000fc0 tx=0x7f6f80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:37.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.766+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6f840ecbd0 con 0x7f6f8c15cb90 2023-10-16T08:16:37.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.938+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f8c06b110 con 0x7f6f8c15cb90 2023-10-16T08:16:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.938+0000 7f6f897fa640 1 -- 172.21.15.32:0/1493737138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f840f01f0 con 0x7f6f8c15cb90 2023-10-16T08:16:37.945 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:16:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f700664a0 msgr2=0x7f6f70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f700664a0 0x7f6f70068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000fc0 tx=0x7f6f80009040 comp rx=0 tx=0).stop 2023-10-16T08:16:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 msgr2=0x7f6f8c159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 0x7f6f8c159780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009d90 tx=0x7f6f84019630 comp rx=0 tx=0).stop 2023-10-16T08:16:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 shutdown_connections 2023-10-16T08:16:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8c15cb90 0x7f6f8c159780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f700664a0 0x7f6f70068960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f8c0d3010 0x7f6f8c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 --2- 172.21.15.32:0/1493737138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f8c0d15d0 0x7f6f8c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 >> 172.21.15.32:0/1493737138 conn(0x7f6f8c09bc50 msgr2=0x7f6f8c15bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:37.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 shutdown_connections 2023-10-16T08:16:37.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:37.942+0000 7f6f92ed6640 1 -- 172.21.15.32:0/1493737138 wait complete. 2023-10-16T08:16:37.968 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:16:37.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:37.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-16T08:16:37.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi032.stderr:ceph-fuse[42907]: fuse finished with error 0 and tester_r 0 2023-10-16T08:16:37.999 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:16:44.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:16:44.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:44.101 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-16T08:16:44.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbaf52a640 1 Processor -- start 2023-10-16T08:16:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbaf52a640 1 -- start start 2023-10-16T08:16:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbaf52a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcba80d3010 0x7fcba80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbaf52a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80ce770 con 0x7fcba80d15d0 2023-10-16T08:16:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbaf52a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80d0600 con 0x7fcba815cb90 2023-10-16T08:16:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbaf52a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba80c6450 con 0x7fcba80d3010 2023-10-16T08:16:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbacac5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcba80d15d0 0x7fcba80c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44346/0 (socket says 172.21.15.32:44346) 2023-10-16T08:16:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbacac5640 1 -- 172.21.15.32:0/3930460796 learned_addr learned my addr 172.21.15.32:0/3930460796 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbadac7640 1 --2- 172.21.15.32:0/3930460796 >> v2:172.21.15.32:3300/0 conn(0x7fcba80d3010 0x7fcba80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbadac7640 1 -- 172.21.15.32:0/3930460796 >> v1:172.21.15.186:6789/0 conn(0x7fcba815cb90 legacy=0x7fcba80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:44.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbadac7640 1 -- 172.21.15.32:0/3930460796 >> v1:172.21.15.32:6789/0 conn(0x7fcba80d15d0 legacy=0x7fcba80c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.318+0000 7fcbadac7640 1 -- 172.21.15.32:0/3930460796 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80d08a0 con 0x7fcba80d3010 2023-10-16T08:16:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbadac7640 1 --2- 172.21.15.32:0/3930460796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d3010 0x7fcba80cfb10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009d90 tx=0x7fcba001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4eda1cc66a41ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcb9ffff640 1 -- 172.21.15.32:0/3930460796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a5910 con 0x7fcba80d3010 2023-10-16T08:16:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcb9ffff640 1 -- 172.21.15.32:0/3930460796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba00a5a70 con 0x7fcba80d3010 2023-10-16T08:16:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcb9ffff640 1 -- 172.21.15.32:0/3930460796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a5e00 con 0x7fcba80d3010 2023-10-16T08:16:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 -- 172.21.15.32:0/3930460796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d3010 msgr2=0x7fcba80cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/3930460796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d3010 0x7fcba80cfb10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009d90 tx=0x7fcba001c850 comp rx=0 tx=0).stop 2023-10-16T08:16:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 -- 172.21.15.32:0/3930460796 shutdown_connections 2023-10-16T08:16:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/3930460796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d3010 0x7fcba80cfb10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 -- 172.21.15.32:0/3930460796 >> 172.21.15.32:0/3930460796 conn(0x7fcba809bc50 msgr2=0x7fcba80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 -- 172.21.15.32:0/3930460796 shutdown_connections 2023-10-16T08:16:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.322+0000 7fcbaf52a640 1 -- 172.21.15.32:0/3930460796 wait complete. 2023-10-16T08:16:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 Processor -- start 2023-10-16T08:16:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- start start 2023-10-16T08:16:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 0x7fcba8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba80d3010 0x7fcba80cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcba815cb90 0x7fcba80cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8146260 con 0x7fcba80d15d0 2023-10-16T08:16:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8159470 con 0x7fcba80d3010 2023-10-16T08:16:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8158d80 con 0x7fcba815cb90 2023-10-16T08:16:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 0x7fcba8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 0x7fcba8158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58878/0 (socket says 172.21.15.32:58878) 2023-10-16T08:16:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 -- 172.21.15.32:0/1819540373 learned_addr learned my addr 172.21.15.32:0/1819540373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbacac5640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba80d3010 0x7fcba80cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbadac7640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcba815cb90 0x7fcba80cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 -- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcba815cb90 msgr2=0x7fcba80cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcba815cb90 0x7fcba80cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 -- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba80d3010 msgr2=0x7fcba80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba80d3010 0x7fcba80cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 -- 172.21.15.32:0/1819540373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0009a70 con 0x7fcba80d15d0 2023-10-16T08:16:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbad2c6640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 0x7fcba8158f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b3f0 tx=0x7fcba400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011830 con 0x7fcba80d15d0 2023-10-16T08:16:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba40119b0 con 0x7fcba80d15d0 2023-10-16T08:16:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba8159680 con 0x7fcba80d15d0 2023-10-16T08:16:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4011d60 con 0x7fcba80d15d0 2023-10-16T08:16:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba8159c40 con 0x7fcba80d15d0 2023-10-16T08:16:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.326+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba8074010 con 0x7fcba80d15d0 2023-10-16T08:16:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.330+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcba401a400 con 0x7fcba80d15d0 2023-10-16T08:16:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.330+0000 7fcb9dffb640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcb8c0664a0 0x7fcb8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.330+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcba4084330 con 0x7fcba80d15d0 2023-10-16T08:16:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.330+0000 7fcbacac5640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcb8c0664a0 0x7fcb8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.330+0000 7fcbacac5640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcb8c0664a0 0x7fcb8c068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcba815aa40 tx=0x7fcb98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.334+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcba4051420 con 0x7fcba80d15d0 2023-10-16T08:16:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.490+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcba806a050 con 0x7fcba80d15d0 2023-10-16T08:16:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.490+0000 7fcb9dffb640 1 -- 172.21.15.32:0/1819540373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fcba4054a40 con 0x7fcba80d15d0 2023-10-16T08:16:44.497 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:44.498 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcb8c0664a0 msgr2=0x7fcb8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcb8c0664a0 0x7fcb8c068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcba815aa40 tx=0x7fcb98009040 comp rx=0 tx=0).stop 2023-10-16T08:16:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 msgr2=0x7fcba8158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 0x7fcba8158f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b3f0 tx=0x7fcba400b8b0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 shutdown_connections 2023-10-16T08:16:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcb8c0664a0 0x7fcb8c068960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcba815cb90 0x7fcba80cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcba80d3010 0x7fcba80cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 --2- 172.21.15.32:0/1819540373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcba80d15d0 0x7fcba8158f30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 >> 172.21.15.32:0/1819540373 conn(0x7fcba809bc50 msgr2=0x7fcba80c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:44.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 shutdown_connections 2023-10-16T08:16:44.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.494+0000 7fcbaf52a640 1 -- 172.21.15.32:0/1819540373 wait complete. 2023-10-16T08:16:44.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.654+0000 7fad68a09640 1 Processor -- start 2023-10-16T08:16:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.654+0000 7fad68a09640 1 -- start start 2023-10-16T08:16:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad68a09640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fad64070c40 0x7fad6415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad68a09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 0x7fad640b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad68a09640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6415be60 con 0x7fad640b0620 2023-10-16T08:16:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad68a09640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6415b3b0 con 0x7fad64070c40 2023-10-16T08:16:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad68a09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad640b0a90 con 0x7fad640b4020 2023-10-16T08:16:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 0x7fad640b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad6259c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fad64070c40 0x7fad6415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 0x7fad640b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36196/0 (socket says 172.21.15.32:36196) 2023-10-16T08:16:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 -- 172.21.15.32:0/1246096048 learned_addr learned my addr 172.21.15.32:0/1246096048 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 -- 172.21.15.32:0/1246096048 >> v2:172.21.15.186:3300/0 conn(0x7fad64070c40 msgr2=0x7fad6415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 --2- 172.21.15.32:0/1246096048 >> v2:172.21.15.186:3300/0 conn(0x7fad64070c40 0x7fad6415ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 -- 172.21.15.32:0/1246096048 >> v1:172.21.15.186:6789/0 conn(0x7fad640b0620 legacy=0x7fad6415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad5bfff640 1 -- 172.21.15.32:0/1246096048 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6415c1f0 con 0x7fad640b4020 2023-10-16T08:16:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.658+0000 7fad6259c640 1 --2- 172.21.15.32:0/1246096048 >> v2:172.21.15.186:3300/0 conn(0x7fad64070c40 0x7fad6415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad5bfff640 1 --2- 172.21.15.32:0/1246096048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 0x7fad640b00e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009870 tx=0x7fad4c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2263fbad1ab0ed95 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad61d9b640 1 -- 172.21.15.32:0/1246096048 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0aa610 con 0x7fad640b4020 2023-10-16T08:16:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad61d9b640 1 -- 172.21.15.32:0/1246096048 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c0a6040 con 0x7fad640b4020 2023-10-16T08:16:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad61d9b640 1 -- 172.21.15.32:0/1246096048 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0aa990 con 0x7fad640b4020 2023-10-16T08:16:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 -- 172.21.15.32:0/1246096048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 msgr2=0x7fad640b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 --2- 172.21.15.32:0/1246096048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 0x7fad640b00e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009870 tx=0x7fad4c01c4b0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 -- 172.21.15.32:0/1246096048 shutdown_connections 2023-10-16T08:16:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 --2- 172.21.15.32:0/1246096048 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad640b4020 0x7fad640b00e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 --2- 172.21.15.32:0/1246096048 >> v2:172.21.15.186:3300/0 conn(0x7fad64070c40 0x7fad6415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 -- 172.21.15.32:0/1246096048 >> 172.21.15.32:0/1246096048 conn(0x7fad6406c190 msgr2=0x7fad6406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 -- 172.21.15.32:0/1246096048 shutdown_connections 2023-10-16T08:16:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 -- 172.21.15.32:0/1246096048 wait complete. 2023-10-16T08:16:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.662+0000 7fad68a09640 1 Processor -- start 2023-10-16T08:16:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- start start 2023-10-16T08:16:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad64070c40 0x7fad641f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad640b0620 0x7fad6414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 0x7fad641f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad641f64a0 con 0x7fad640b4020 2023-10-16T08:16:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad641f6740 con 0x7fad640b0620 2023-10-16T08:16:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad641f6a20 con 0x7fad64070c40 2023-10-16T08:16:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 0x7fad641f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 0x7fad641f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58888/0 (socket says 172.21.15.32:58888) 2023-10-16T08:16:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 -- 172.21.15.32:0/647177820 learned_addr learned my addr 172.21.15.32:0/647177820 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad5bfff640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad640b0620 0x7fad6414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad6259c640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad64070c40 0x7fad641f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 -- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad64070c40 msgr2=0x7fad641f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad64070c40 0x7fad641f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 -- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad640b0620 msgr2=0x7fad6414ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad640b0620 0x7fad6414ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 -- 172.21.15.32:0/647177820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad54009420 con 0x7fad640b4020 2023-10-16T08:16:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad5bfff640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad640b0620 0x7fad6414ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad62d9d640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 0x7fad641f31a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c6c0 tx=0x7fad5c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c003b80 con 0x7fad640b4020 2023-10-16T08:16:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c003ce0 con 0x7fad640b4020 2023-10-16T08:16:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0052f0 con 0x7fad640b4020 2023-10-16T08:16:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c0094d0 con 0x7fad640b4020 2023-10-16T08:16:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad641f2c80 con 0x7fad640b4020 2023-10-16T08:16:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.666+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad641f2e70 con 0x7fad640b4020 2023-10-16T08:16:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.670+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad5c005450 con 0x7fad640b4020 2023-10-16T08:16:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.670+0000 7fad5affd640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fad440664a0 0x7fad44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.670+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad5c083dc0 con 0x7fad640b4020 2023-10-16T08:16:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.670+0000 7fad6259c640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fad440664a0 0x7fad44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.670+0000 7fad6259c640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fad440664a0 0x7fad44068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad54009cf0 tx=0x7fad54009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.674+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad5c050ff0 con 0x7fad640b4020 2023-10-16T08:16:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad64074010 con 0x7fad640b4020 2023-10-16T08:16:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad5affd640 1 -- 172.21.15.32:0/647177820 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fad5c054610 con 0x7fad640b4020 2023-10-16T08:16:44.840 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fad440664a0 msgr2=0x7fad44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fad440664a0 0x7fad44068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad54009cf0 tx=0x7fad54009190 comp rx=0 tx=0).stop 2023-10-16T08:16:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 msgr2=0x7fad641f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 0x7fad641f31a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00c6c0 tx=0x7fad5c00cb80 comp rx=0 tx=0).stop 2023-10-16T08:16:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 shutdown_connections 2023-10-16T08:16:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad640b4020 0x7fad641f31a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad640b0620 0x7fad6414ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fad440664a0 0x7fad44068960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 --2- 172.21.15.32:0/647177820 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fad64070c40 0x7fad641f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.834+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 >> 172.21.15.32:0/647177820 conn(0x7fad6406c190 msgr2=0x7fad640afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.838+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 shutdown_connections 2023-10-16T08:16:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:44.838+0000 7fad68a09640 1 -- 172.21.15.32:0/647177820 wait complete. 2023-10-16T08:16:44.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T08:16:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 Processor -- start 2023-10-16T08:16:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 -- start start 2023-10-16T08:16:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57100d0210 0x7f57100d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f57100b84e0 0x7f57100b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100d1670 con 0x7f571015cba0 2023-10-16T08:16:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100d1460 con 0x7f57100d0210 2023-10-16T08:16:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f571559d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100d1950 con 0x7f57100b84e0 2023-10-16T08:16:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f5707fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57100d0210 0x7f57100d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f570effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f571015cba0 0x7f57100cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44356/0 (socket says 172.21.15.32:44356) 2023-10-16T08:16:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f570effd640 1 -- 172.21.15.32:0/1746076596 learned_addr learned my addr 172.21.15.32:0/1746076596 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f570f7fe640 1 --2- 172.21.15.32:0/1746076596 >> v2:172.21.15.186:3300/0 conn(0x7f57100b84e0 0x7f57100b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f5707fff640 1 -- 172.21.15.32:0/1746076596 >> v2:172.21.15.186:3300/0 conn(0x7f57100b84e0 msgr2=0x7f57100b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f5707fff640 1 --2- 172.21.15.32:0/1746076596 >> v2:172.21.15.186:3300/0 conn(0x7f57100b84e0 0x7f57100b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f5707fff640 1 -- 172.21.15.32:0/1746076596 >> v1:172.21.15.32:6789/0 conn(0x7f571015cba0 legacy=0x7f57100cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f5707fff640 1 -- 172.21.15.32:0/1746076596 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100d1c30 con 0x7f57100d0210 2023-10-16T08:16:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.002+0000 7f570e7fc640 1 -- 172.21.15.32:0/1746076596 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193010781 0 0) 0x7f57100d1670 con 0x7f571015cba0 2023-10-16T08:16:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f5707fff640 1 --2- 172.21.15.32:0/1746076596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f57100d0f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57100ce450 tx=0x7f56f801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5803904ec3e184b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f570e7fc640 1 -- 172.21.15.32:0/1746076596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80ac630 con 0x7f57100d0210 2023-10-16T08:16:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f570e7fc640 1 -- 172.21.15.32:0/1746076596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f80ac790 con 0x7f57100d0210 2023-10-16T08:16:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f570e7fc640 1 -- 172.21.15.32:0/1746076596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80acb20 con 0x7f57100d0210 2023-10-16T08:16:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 -- 172.21.15.32:0/1746076596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 msgr2=0x7f57100d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 --2- 172.21.15.32:0/1746076596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f57100d0f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57100ce450 tx=0x7f56f801c3d0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 -- 172.21.15.32:0/1746076596 shutdown_connections 2023-10-16T08:16:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 --2- 172.21.15.32:0/1746076596 >> v2:172.21.15.186:3300/0 conn(0x7f57100b84e0 0x7f57100b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 --2- 172.21.15.32:0/1746076596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f57100d0f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 -- 172.21.15.32:0/1746076596 >> 172.21.15.32:0/1746076596 conn(0x7f57100719c0 msgr2=0x7f5710071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 -- 172.21.15.32:0/1746076596 shutdown_connections 2023-10-16T08:16:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.006+0000 7f571559d640 1 -- 172.21.15.32:0/1746076596 wait complete. 2023-10-16T08:16:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 Processor -- start 2023-10-16T08:16:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- start start 2023-10-16T08:16:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57100b84e0 0x7f571015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f571014a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f571015cba0 0x7f5710147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710155f10 con 0x7f57100d0210 2023-10-16T08:16:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571014aea0 con 0x7f57100b84e0 2023-10-16T08:16:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571014b180 con 0x7f571015cba0 2023-10-16T08:16:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f571014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f571014a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58896/0 (socket says 172.21.15.32:58896) 2023-10-16T08:16:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 -- 172.21.15.32:0/1306850346 learned_addr learned my addr 172.21.15.32:0/1306850346 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f570effd640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57100b84e0 0x7f571015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f570f7fe640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f571015cba0 0x7f5710147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 -- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f571015cba0 msgr2=0x7f5710147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f571015cba0 0x7f5710147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 -- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57100b84e0 msgr2=0x7f571015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57100b84e0 0x7f571015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 -- 172.21.15.32:0/1306850346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009420 con 0x7f57100d0210 2023-10-16T08:16:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f5707fff640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f571014a670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f56f8018fe0 tx=0x7f56f8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80b5600 con 0x7f57100d0210 2023-10-16T08:16:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f80b5760 con 0x7f57100d0210 2023-10-16T08:16:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80b5af0 con 0x7f57100d0210 2023-10-16T08:16:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571014b460 con 0x7f57100d0210 2023-10-16T08:16:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571014b740 con 0x7f57100d0210 2023-10-16T08:16:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.010+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571004e170 con 0x7f57100d0210 2023-10-16T08:16:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.014+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56f80b3040 con 0x7f57100d0210 2023-10-16T08:16:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.014+0000 7f57077fe640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f56f00664d0 0x7f56f0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.014+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f56f811e510 con 0x7f57100d0210 2023-10-16T08:16:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.014+0000 7f570effd640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f56f00664d0 0x7f56f0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.014+0000 7f570effd640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f56f00664d0 0x7f56f0068990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5700000f70 tx=0x7f5700007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.018+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f56f80eb740 con 0x7f57100d0210 2023-10-16T08:16:45.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.210+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5710154ab0 con 0x7f57100d0210 2023-10-16T08:16:45.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f57077fe640 1 -- 172.21.15.32:0/1306850346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56f80b5e60 con 0x7f57100d0210 2023-10-16T08:16:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f56f00664d0 msgr2=0x7f56f0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f56f00664d0 0x7f56f0068990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5700000f70 tx=0x7f5700007df0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 msgr2=0x7f571014a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f571014a670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f56f8018fe0 tx=0x7f56f8019170 comp rx=0 tx=0).stop 2023-10-16T08:16:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 shutdown_connections 2023-10-16T08:16:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f56f00664d0 0x7f56f0068990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f571015cba0 0x7f5710147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57100d0210 0x7f571014a670 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 --2- 172.21.15.32:0/1306850346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57100b84e0 0x7f571015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 >> 172.21.15.32:0/1306850346 conn(0x7f57100719c0 msgr2=0x7f5710153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 shutdown_connections 2023-10-16T08:16:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.770+0000 7f571559d640 1 -- 172.21.15.32:0/1306850346 wait complete. 2023-10-16T08:16:45.801 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-16T08:16:45.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T08:16:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 Processor -- start 2023-10-16T08:16:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 -- start start 2023-10-16T08:16:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0b240ade00 0x7f0b240c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b2406a4d0 0x7f0b240ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2406c6d0 con 0x7f0b2406a0f0 2023-10-16T08:16:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2406c950 con 0x7f0b240ade00 2023-10-16T08:16:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b29b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241654f0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b237fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b2406a0f0 0x7f0b240ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44362/0 (socket says 172.21.15.32:44362) 2023-10-16T08:16:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b237fe640 1 -- 172.21.15.32:0/2736260264 learned_addr learned my addr 172.21.15.32:0/2736260264 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b22ffd640 1 --2- 172.21.15.32:0/2736260264 >> v2:172.21.15.186:3300/0 conn(0x7f0b240ade00 0x7f0b240c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b23fff640 1 --2- 172.21.15.32:0/2736260264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b2406a4d0 0x7f0b240ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b227fc640 1 -- 172.21.15.32:0/2736260264 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289798075 0 0) 0x7f0b2406c6d0 con 0x7f0b2406a0f0 2023-10-16T08:16:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b227fc640 1 -- 172.21.15.32:0/2736260264 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c003530 con 0x7f0b2406a0f0 2023-10-16T08:16:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b23fff640 1 -- 172.21.15.32:0/2736260264 >> v2:172.21.15.186:3300/0 conn(0x7f0b240ade00 msgr2=0x7f0b240c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b23fff640 1 --2- 172.21.15.32:0/2736260264 >> v2:172.21.15.186:3300/0 conn(0x7f0b240ade00 0x7f0b240c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b23fff640 1 -- 172.21.15.32:0/2736260264 >> v1:172.21.15.32:6789/0 conn(0x7f0b2406a0f0 legacy=0x7f0b240ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b23fff640 1 -- 172.21.15.32:0/2736260264 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b241657d0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.946+0000 7f0b22ffd640 1 --2- 172.21.15.32:0/2736260264 >> v2:172.21.15.186:3300/0 conn(0x7f0b240ade00 0x7f0b240c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b23fff640 1 --2- 172.21.15.32:0/2736260264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b2406a4d0 0x7f0b240ccd00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009d90 tx=0x7f0b1c01d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b0e4dc3242cf7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b227fc640 1 -- 172.21.15.32:0/2736260264 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0a5a30 con 0x7f0b2406a4d0 2023-10-16T08:16:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b227fc640 1 -- 172.21.15.32:0/2736260264 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c0a5b90 con 0x7f0b2406a4d0 2023-10-16T08:16:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b227fc640 1 -- 172.21.15.32:0/2736260264 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0ad440 con 0x7f0b2406a4d0 2023-10-16T08:16:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 -- 172.21.15.32:0/2736260264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b2406a4d0 msgr2=0x7f0b240ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/2736260264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b2406a4d0 0x7f0b240ccd00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009d90 tx=0x7f0b1c01d3b0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 -- 172.21.15.32:0/2736260264 shutdown_connections 2023-10-16T08:16:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/2736260264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b2406a4d0 0x7f0b240ccd00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/2736260264 >> v2:172.21.15.186:3300/0 conn(0x7f0b240ade00 0x7f0b240c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 -- 172.21.15.32:0/2736260264 >> 172.21.15.32:0/2736260264 conn(0x7f0b240b0da0 msgr2=0x7f0b240ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:45.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 -- 172.21.15.32:0/2736260264 shutdown_connections 2023-10-16T08:16:45.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.950+0000 7f0b29b3e640 1 -- 172.21.15.32:0/2736260264 wait complete. 2023-10-16T08:16:45.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 Processor -- start 2023-10-16T08:16:45.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 -- start start 2023-10-16T08:16:45.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b2406a0f0 0x7f0b240b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 0x7f0b24158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b240ade00 0x7f0b240b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241594a0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b1a00 con 0x7f0b2406a0f0 2023-10-16T08:16:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b29b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b1ce0 con 0x7f0b240ade00 2023-10-16T08:16:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 0x7f0b24158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 0x7f0b24158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58910/0 (socket says 172.21.15.32:58910) 2023-10-16T08:16:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 -- 172.21.15.32:0/880783472 learned_addr learned my addr 172.21.15.32:0/880783472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b23fff640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b240ade00 0x7f0b240b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b237fe640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b2406a0f0 0x7f0b240b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 -- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b240ade00 msgr2=0x7f0b240b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b240ade00 0x7f0b240b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 -- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b2406a0f0 msgr2=0x7f0b240b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b2406a0f0 0x7f0b240b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:45.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 -- 172.21.15.32:0/880783472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18009420 con 0x7f0b2406a4d0 2023-10-16T08:16:45.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b237fe640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b2406a0f0 0x7f0b240b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:45.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.954+0000 7f0b22ffd640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 0x7f0b24158f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800ebb0 tx=0x7f0b1800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:45.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800eeb0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1800cc30 con 0x7f0b2406a4d0 2023-10-16T08:16:45.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18013470 con 0x7f0b2406a4d0 2023-10-16T08:16:45.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c009a70 con 0x7f0b2406a4d0 2023-10-16T08:16:45.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240b22a0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2404e170 con 0x7f0b2406a4d0 2023-10-16T08:16:45.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0b180135d0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b0bfff640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0b100664d0 0x7f0b10068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:45.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0b180843d0 con 0x7f0b2406a4d0 2023-10-16T08:16:45.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.958+0000 7f0b237fe640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0b100664d0 0x7f0b10068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:45.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.962+0000 7f0b237fe640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0b100664d0 0x7f0b10068990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005890 tx=0x7f0b14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:45.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:45.962+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0b18051570 con 0x7f0b2406a4d0 2023-10-16T08:16:46.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.154+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0b24164060 con 0x7f0b2406a4d0 2023-10-16T08:16:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.774+0000 7f0b0bfff640 1 -- 172.21.15.32:0/880783472 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b18054b90 con 0x7f0b2406a4d0 2023-10-16T08:16:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0b100664d0 msgr2=0x7f0b10068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0b100664d0 0x7f0b10068990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005890 tx=0x7f0b14009040 comp rx=0 tx=0).stop 2023-10-16T08:16:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 msgr2=0x7f0b24158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 0x7f0b24158f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800ebb0 tx=0x7f0b1800c520 comp rx=0 tx=0).stop 2023-10-16T08:16:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 shutdown_connections 2023-10-16T08:16:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0b100664d0 0x7f0b10068990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b240ade00 0x7f0b240b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b2406a4d0 0x7f0b24158f60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 --2- 172.21.15.32:0/880783472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b2406a0f0 0x7f0b240b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 >> 172.21.15.32:0/880783472 conn(0x7f0b240b0da0 msgr2=0x7f0b24072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 shutdown_connections 2023-10-16T08:16:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.778+0000 7f0b29b3e640 1 -- 172.21.15.32:0/880783472 wait complete. 2023-10-16T08:16:46.807 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T08:16:46.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:46.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.950+0000 7f46a7af1640 1 Processor -- start 2023-10-16T08:16:46.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.950+0000 7f46a7af1640 1 -- start start 2023-10-16T08:16:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a7af1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00d15d0 0x7f46a00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a7af1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f46a00d3010 0x7f46a00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a7af1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00ce770 con 0x7f46a015cb90 2023-10-16T08:16:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a7af1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00d0600 con 0x7f46a00d15d0 2023-10-16T08:16:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a7af1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00c6450 con 0x7f46a00d3010 2023-10-16T08:16:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00d15d0 0x7f46a00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00d15d0 0x7f46a00cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58924/0 (socket says 172.21.15.32:58924) 2023-10-16T08:16:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 -- 172.21.15.32:0/2289064768 learned_addr learned my addr 172.21.15.32:0/2289064768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a608e640 1 --2- 172.21.15.32:0/2289064768 >> v2:172.21.15.186:3300/0 conn(0x7f46a00d3010 0x7f46a00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 -- 172.21.15.32:0/2289064768 >> v2:172.21.15.186:3300/0 conn(0x7f46a00d3010 msgr2=0x7f46a00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 --2- 172.21.15.32:0/2289064768 >> v2:172.21.15.186:3300/0 conn(0x7f46a00d3010 0x7f46a00c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 -- 172.21.15.32:0/2289064768 >> v1:172.21.15.186:6789/0 conn(0x7f46a015cb90 legacy=0x7f46a00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.954+0000 7f46a508c640 1 -- 172.21.15.32:0/2289064768 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00d08a0 con 0x7f46a00d15d0 2023-10-16T08:16:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a508c640 1 --2- 172.21.15.32:0/2289064768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00d15d0 0x7f46a00cfb10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f46a0072590 tx=0x7f469001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c61eba9efe6cdf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a488b640 1 -- 172.21.15.32:0/2289064768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900ab640 con 0x7f46a00d15d0 2023-10-16T08:16:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a488b640 1 -- 172.21.15.32:0/2289064768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46900a7070 con 0x7f46a00d15d0 2023-10-16T08:16:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a488b640 1 -- 172.21.15.32:0/2289064768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900ab9c0 con 0x7f46a00d15d0 2023-10-16T08:16:46.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 -- 172.21.15.32:0/2289064768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00d15d0 msgr2=0x7f46a00cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:46.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 --2- 172.21.15.32:0/2289064768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00d15d0 0x7f46a00cfb10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f46a0072590 tx=0x7f469001c440 comp rx=0 tx=0).stop 2023-10-16T08:16:46.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 -- 172.21.15.32:0/2289064768 shutdown_connections 2023-10-16T08:16:46.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 --2- 172.21.15.32:0/2289064768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00d15d0 0x7f46a00cfb10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 --2- 172.21.15.32:0/2289064768 >> v2:172.21.15.186:3300/0 conn(0x7f46a00d3010 0x7f46a00c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 -- 172.21.15.32:0/2289064768 >> 172.21.15.32:0/2289064768 conn(0x7f46a009bc50 msgr2=0x7f46a00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:46.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 -- 172.21.15.32:0/2289064768 shutdown_connections 2023-10-16T08:16:46.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.958+0000 7f46a7af1640 1 -- 172.21.15.32:0/2289064768 wait complete. 2023-10-16T08:16:46.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 Processor -- start 2023-10-16T08:16:46.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- start start 2023-10-16T08:16:46.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46a00d15d0 0x7f46a006bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:46.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a00d3010 0x7f46a00cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:46.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 0x7f46a00cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:46.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0157ba0 con 0x7f46a015cb90 2023-10-16T08:16:46.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0157e80 con 0x7f46a00d3010 2023-10-16T08:16:46.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0158160 con 0x7f46a00d15d0 2023-10-16T08:16:46.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 0x7f46a00cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:46.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 0x7f46a00cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58930/0 (socket says 172.21.15.32:58930) 2023-10-16T08:16:46.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 -- 172.21.15.32:0/1091532938 learned_addr learned my addr 172.21.15.32:0/1091532938 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:46.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a508c640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a00d3010 0x7f46a00cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:46.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a588d640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46a00d15d0 0x7f46a006bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:46.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 -- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46a00d15d0 msgr2=0x7f46a006bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:46.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46a00d15d0 0x7f46a006bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 -- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a00d3010 msgr2=0x7f46a00cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:46.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a00d3010 0x7f46a00cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:46.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 -- 172.21.15.32:0/1091532938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46900094d0 con 0x7f46a015cb90 2023-10-16T08:16:46.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a508c640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a00d3010 0x7f46a00cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:46.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a608e640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 0x7f46a00cbf30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f469800efc0 tx=0x7f469800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698014480 con 0x7f46a015cb90 2023-10-16T08:16:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800f040 con 0x7f46a015cb90 2023-10-16T08:16:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698014820 con 0x7f46a015cb90 2023-10-16T08:16:46.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a009e2c0 con 0x7f46a015cb90 2023-10-16T08:16:46.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a009c890 con 0x7f46a015cb90 2023-10-16T08:16:46.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.962+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0158380 con 0x7f46a015cb90 2023-10-16T08:16:46.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.966+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4698014980 con 0x7f46a015cb90 2023-10-16T08:16:46.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.966+0000 7f46967fc640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46840664d0 0x7f4684068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:46.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.966+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4698084d50 con 0x7f46a015cb90 2023-10-16T08:16:46.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.966+0000 7f46a588d640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46840664d0 0x7f4684068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:46.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.966+0000 7f46a588d640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46840664d0 0x7f4684068990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f469c000fc0 tx=0x7f469c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:46.970+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4698051e40 con 0x7f46a015cb90 2023-10-16T08:16:47.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:16:47.083 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:16:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46a0074010 con 0x7f46a015cb90 2023-10-16T08:16:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46967fc640 1 -- 172.21.15.32:0/1091532938 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f4698018020 con 0x7f46a015cb90 2023-10-16T08:16:47.167 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:47.167 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:16:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:16:47.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46840664d0 msgr2=0x7f4684068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46840664d0 0x7f4684068990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f469c000fc0 tx=0x7f469c009040 comp rx=0 tx=0).stop 2023-10-16T08:16:47.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 msgr2=0x7f46a00cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 0x7f46a00cbf30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f469800efc0 tx=0x7f469800c490 comp rx=0 tx=0).stop 2023-10-16T08:16:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 shutdown_connections 2023-10-16T08:16:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a015cb90 0x7f46a00cbf30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46a00d3010 0x7f46a00cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46840664d0 0x7f4684068990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 --2- 172.21.15.32:0/1091532938 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46a00d15d0 0x7f46a006bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 >> 172.21.15.32:0/1091532938 conn(0x7f46a009bc50 msgr2=0x7f46a00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 shutdown_connections 2023-10-16T08:16:47.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.162+0000 7f46a7af1640 1 -- 172.21.15.32:0/1091532938 wait complete. 2023-10-16T08:16:47.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 Processor -- start 2023-10-16T08:16:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 -- start start 2023-10-16T08:16:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd4c0cf090 0x7efd4c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efd4c0c6ad0 0x7efd4c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4c0c65f0 con 0x7efd4c15ca70 2023-10-16T08:16:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c0cfa60 con 0x7efd4c0cf090 2023-10-16T08:16:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd5149e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c0ce820 con 0x7efd4c0c6ad0 2023-10-16T08:16:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd43fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd4c0cf090 0x7efd4c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd43fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd4c0cf090 0x7efd4c0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58936/0 (socket says 172.21.15.32:58936) 2023-10-16T08:16:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.366+0000 7efd43fff640 1 -- 172.21.15.32:0/1825396325 learned_addr learned my addr 172.21.15.32:0/1825396325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd4b7fe640 1 --2- 172.21.15.32:0/1825396325 >> v2:172.21.15.186:3300/0 conn(0x7efd4c0c6ad0 0x7efd4c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd43fff640 1 -- 172.21.15.32:0/1825396325 >> v2:172.21.15.186:3300/0 conn(0x7efd4c0c6ad0 msgr2=0x7efd4c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd43fff640 1 --2- 172.21.15.32:0/1825396325 >> v2:172.21.15.186:3300/0 conn(0x7efd4c0c6ad0 0x7efd4c0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd43fff640 1 -- 172.21.15.32:0/1825396325 >> v1:172.21.15.32:6789/0 conn(0x7efd4c15ca70 legacy=0x7efd4c0d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd43fff640 1 -- 172.21.15.32:0/1825396325 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c0cdf80 con 0x7efd4c0cf090 2023-10-16T08:16:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd4a7fc640 1 -- 172.21.15.32:0/1825396325 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112891848 0 0) 0x7efd4c0c65f0 con 0x7efd4c15ca70 2023-10-16T08:16:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd43fff640 1 --2- 172.21.15.32:0/1825396325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c0cf520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0b8950 tx=0x7efd3401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9efd241f80d68 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd4a7fc640 1 -- 172.21.15.32:0/1825396325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340aa430 con 0x7efd4c0cf090 2023-10-16T08:16:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd4a7fc640 1 -- 172.21.15.32:0/1825396325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340aa590 con 0x7efd4c0cf090 2023-10-16T08:16:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd4a7fc640 1 -- 172.21.15.32:0/1825396325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340aa920 con 0x7efd4c0cf090 2023-10-16T08:16:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd5149e640 1 -- 172.21.15.32:0/1825396325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 msgr2=0x7efd4c0cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd5149e640 1 --2- 172.21.15.32:0/1825396325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c0cf520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0b8950 tx=0x7efd3401c4d0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd5149e640 1 -- 172.21.15.32:0/1825396325 shutdown_connections 2023-10-16T08:16:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd5149e640 1 --2- 172.21.15.32:0/1825396325 >> v2:172.21.15.186:3300/0 conn(0x7efd4c0c6ad0 0x7efd4c0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd5149e640 1 --2- 172.21.15.32:0/1825396325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c0cf520 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.370+0000 7efd5149e640 1 -- 172.21.15.32:0/1825396325 >> 172.21.15.32:0/1825396325 conn(0x7efd4c09bc50 msgr2=0x7efd4c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- 172.21.15.32:0/1825396325 shutdown_connections 2023-10-16T08:16:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- 172.21.15.32:0/1825396325 wait complete. 2023-10-16T08:16:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 Processor -- start 2023-10-16T08:16:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- start start 2023-10-16T08:16:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd4c0c6ad0 0x7efd4c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4c15ca70 0x7efd4c09c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c15b140 con 0x7efd4c0cf090 2023-10-16T08:16:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c0cadb0 con 0x7efd4c15ca70 2023-10-16T08:16:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c0caad0 con 0x7efd4c0c6ad0 2023-10-16T08:16:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58950/0 (socket says 172.21.15.32:58950) 2023-10-16T08:16:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 -- 172.21.15.32:0/3761581006 learned_addr learned my addr 172.21.15.32:0/3761581006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 -- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd4c0c6ad0 msgr2=0x7efd4c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd4c0c6ad0 0x7efd4c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 -- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4c15ca70 msgr2=0x7efd4c09c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4c15ca70 0x7efd4c09c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 -- 172.21.15.32:0/3761581006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340094d0 con 0x7efd4c0cf090 2023-10-16T08:16:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd43fff640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c15b5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd3401c9b0 tx=0x7efd3401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340b3800 con 0x7efd4c0cf090 2023-10-16T08:16:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340b3980 con 0x7efd4c0cf090 2023-10-16T08:16:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c0d38a0 con 0x7efd4c0cf090 2023-10-16T08:16:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340b3d30 con 0x7efd4c0cf090 2023-10-16T08:16:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.374+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c15add0 con 0x7efd4c0cf090 2023-10-16T08:16:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.378+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c074010 con 0x7efd4c0cf090 2023-10-16T08:16:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.378+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd340c6390 con 0x7efd4c0cf090 2023-10-16T08:16:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.378+0000 7efd437fe640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efd2c0664a0 0x7efd2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.378+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efd3411ea10 con 0x7efd4c0cf090 2023-10-16T08:16:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.378+0000 7efd4affd640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efd2c0664a0 0x7efd2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.378+0000 7efd4affd640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efd2c0664a0 0x7efd2c068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd3c007fd0 tx=0x7efd3c007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.382+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd340ebb90 con 0x7efd4c0cf090 2023-10-16T08:16:47.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.542+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd4c15b2f0 con 0x7efd4c0cf090 2023-10-16T08:16:47.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.542+0000 7efd437fe640 1 -- 172.21.15.32:0/3761581006 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7efd4c15b2f0 con 0x7efd4c0cf090 2023-10-16T08:16:47.548 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:47.548 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efd2c0664a0 msgr2=0x7efd2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efd2c0664a0 0x7efd2c068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd3c007fd0 tx=0x7efd3c007980 comp rx=0 tx=0).stop 2023-10-16T08:16:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 msgr2=0x7efd4c15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c15b5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd3401c9b0 tx=0x7efd3401ad10 comp rx=0 tx=0).stop 2023-10-16T08:16:47.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 shutdown_connections 2023-10-16T08:16:47.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd4c15ca70 0x7efd4c09c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd4c0cf090 0x7efd4c15b5d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efd2c0664a0 0x7efd2c068960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 --2- 172.21.15.32:0/3761581006 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efd4c0c6ad0 0x7efd4c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 >> 172.21.15.32:0/3761581006 conn(0x7efd4c09bc50 msgr2=0x7efd4c06bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:47.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 shutdown_connections 2023-10-16T08:16:47.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.546+0000 7efd5149e640 1 -- 172.21.15.32:0/3761581006 wait complete. 2023-10-16T08:16:47.570 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T08:16:47.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:16:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.706+0000 7f33d5bc4640 1 Processor -- start 2023-10-16T08:16:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.706+0000 7f33d5bc4640 1 -- start start 2023-10-16T08:16:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.706+0000 7f33d5bc4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f33d015ca70 0x7f33d00d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.706+0000 7f33d5bc4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d00c65f0 con 0x7f33d00cf7a0 2023-10-16T08:16:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.706+0000 7f33d5bc4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d00cdf80 con 0x7f33d00d3550 2023-10-16T08:16:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.706+0000 7f33d5bc4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d00ce820 con 0x7f33d015ca70 2023-10-16T08:16:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33d00cf7a0 0x7f33d00c6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44388/0 (socket says 172.21.15.32:44388) 2023-10-16T08:16:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cffff640 1 -- 172.21.15.32:0/2008458950 learned_addr learned my addr 172.21.15.32:0/2008458950 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cf7fe640 1 --2- 172.21.15.32:0/2008458950 >> v2:172.21.15.186:3300/0 conn(0x7f33d015ca70 0x7f33d00d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33ce7fc640 1 -- 172.21.15.32:0/2008458950 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418623334 0 0) 0x7f33d00c65f0 con 0x7f33d00cf7a0 2023-10-16T08:16:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33ce7fc640 1 -- 172.21.15.32:0/2008458950 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b8003530 con 0x7f33d00cf7a0 2023-10-16T08:16:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cf7fe640 1 -- 172.21.15.32:0/2008458950 >> v1:172.21.15.186:6789/0 conn(0x7f33d00d3550 legacy=0x7f33d00cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cf7fe640 1 -- 172.21.15.32:0/2008458950 >> v1:172.21.15.32:6789/0 conn(0x7f33d00cf7a0 legacy=0x7f33d00c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cf7fe640 1 -- 172.21.15.32:0/2008458950 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00ce260 con 0x7f33d015ca70 2023-10-16T08:16:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33cf7fe640 1 --2- 172.21.15.32:0/2008458950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d015ca70 0x7f33d00d3010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33c00097a0 tx=0x7f33c001c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2103887c5a9ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33ce7fc640 1 -- 172.21.15.32:0/2008458950 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c00a8050 con 0x7f33d015ca70 2023-10-16T08:16:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33ce7fc640 1 -- 172.21.15.32:0/2008458950 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c00ac5f0 con 0x7f33d015ca70 2023-10-16T08:16:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.710+0000 7f33ce7fc640 1 -- 172.21.15.32:0/2008458950 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c00ac980 con 0x7f33d015ca70 2023-10-16T08:16:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- 172.21.15.32:0/2008458950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d015ca70 msgr2=0x7f33d00d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/2008458950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d015ca70 0x7f33d00d3010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33c00097a0 tx=0x7f33c001c430 comp rx=0 tx=0).stop 2023-10-16T08:16:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- 172.21.15.32:0/2008458950 shutdown_connections 2023-10-16T08:16:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/2008458950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d015ca70 0x7f33d00d3010 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- 172.21.15.32:0/2008458950 >> 172.21.15.32:0/2008458950 conn(0x7f33d009bc50 msgr2=0x7f33d00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- 172.21.15.32:0/2008458950 shutdown_connections 2023-10-16T08:16:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- 172.21.15.32:0/2008458950 wait complete. 2023-10-16T08:16:47.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 Processor -- start 2023-10-16T08:16:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- start start 2023-10-16T08:16:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33d00cf7a0 0x7f33d01567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d00d3550 0x7f33d00cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 0x7f33d00cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0159bd0 con 0x7f33d015ca70 2023-10-16T08:16:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d01592d0 con 0x7f33d00d3550 2023-10-16T08:16:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33d5bc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00cae00 con 0x7f33d00cf7a0 2023-10-16T08:16:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 0x7f33d00cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33cf7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33d00cf7a0 0x7f33d01567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 0x7f33d00cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58964/0 (socket says 172.21.15.32:58964) 2023-10-16T08:16:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33cffff640 1 -- 172.21.15.32:0/970176470 learned_addr learned my addr 172.21.15.32:0/970176470 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.714+0000 7f33ceffd640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d00d3550 0x7f33d00cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cffff640 1 -- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33d00cf7a0 msgr2=0x7f33d01567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cffff640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33d00cf7a0 0x7f33d01567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cffff640 1 -- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d00d3550 msgr2=0x7f33d00cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cffff640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d00d3550 0x7f33d00cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cffff640 1 -- 172.21.15.32:0/970176470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00093b0 con 0x7f33d015ca70 2023-10-16T08:16:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cffff640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 0x7f33d00cbe50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f33c800cbb0 tx=0x7f33c800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c800ace0 con 0x7f33d015ca70 2023-10-16T08:16:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c800ae40 con 0x7f33d015ca70 2023-10-16T08:16:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8012600 con 0x7f33d015ca70 2023-10-16T08:16:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d0159510 con 0x7f33d015ca70 2023-10-16T08:16:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d0156d30 con 0x7f33d015ca70 2023-10-16T08:16:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d0159d10 con 0x7f33d015ca70 2023-10-16T08:16:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f33c8012820 con 0x7f33d015ca70 2023-10-16T08:16:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33b7fff640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33bc0664a0 0x7f33bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f33c80840b0 con 0x7f33d015ca70 2023-10-16T08:16:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.718+0000 7f33cf7fe640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33bc0664a0 0x7f33bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.722+0000 7f33cf7fe640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33bc0664a0 0x7f33bc068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ae000 tx=0x7f33c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.722+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f33d0159d10 con 0x7f33d015ca70 2023-10-16T08:16:47.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.878+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33d00c79f0 con 0x7f33bc0664a0 2023-10-16T08:16:47.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.890+0000 7f33b7fff640 1 -- 172.21.15.32:0/970176470 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345290 (secure 0 0 0) 0x7f33c00ac860 con 0x7f33bc0664a0 2023-10-16T08:16:47.899 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:47.901 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:16:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33bc0664a0 msgr2=0x7f33bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33bc0664a0 0x7f33bc068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ae000 tx=0x7f33c0008040 comp rx=0 tx=0).stop 2023-10-16T08:16:47.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 msgr2=0x7f33d00cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 0x7f33d00cbe50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f33c800cbb0 tx=0x7f33c800a430 comp rx=0 tx=0).stop 2023-10-16T08:16:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 shutdown_connections 2023-10-16T08:16:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d015ca70 0x7f33d00cbe50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33d00d3550 0x7f33d00cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33bc0664a0 0x7f33bc068960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 --2- 172.21.15.32:0/970176470 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33d00cf7a0 0x7f33d01567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 >> 172.21.15.32:0/970176470 conn(0x7f33d009bc50 msgr2=0x7f33d0072920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 shutdown_connections 2023-10-16T08:16:47.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:47.894+0000 7f33d5bc4640 1 -- 172.21.15.32:0/970176470 wait complete. 2023-10-16T08:16:47.919 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-16T08:16:46.019007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223628,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751092,"statfs":{"total":773094113280,"available":772865118208,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":225496635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003280"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988248+0000","last_change":"2023-10-16T08:16:21.956684+0000","last_active":"2023-10-16T08:16:26.988248+0000","last_peered":"2023-10-16T08:16:26.988248+0000","last_clean":"2023-10-16T08:16:26.988248+0000","last_became_active":"2023-10-16T08:16:21.949061+0000","last_became_peered":"2023-10-16T08:16:21.949061+0000","last_unstale":"2023-10-16T08:16:26.988248+0000","last_undegraded":"2023-10-16T08:16:26.988248+0000","last_fullsized":"2023-10-16T08:16:26.988248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985413+0000","last_change":"2023-10-16T08:16:22.972913+0000","last_active":"2023-10-16T08:16:26.985413+0000","last_peered":"2023-10-16T08:16:26.985413+0000","last_clean":"2023-10-16T08:16:26.985413+0000","last_became_active":"2023-10-16T08:16:22.972605+0000","last_became_peered":"2023-10-16T08:16:22.972605+0000","last_unstale":"2023-10-16T08:16:26.985413+0000","last_undegraded":"2023-10-16T08:16:26.985413+0000","last_fullsized":"2023-10-16T08:16:26.985413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989573+0000","last_change":"2023-10-16T08:16:27.182406+0000","last_active":"2023-10-16T08:16:27.989573+0000","last_peered":"2023-10-16T08:16:27.989573+0000","last_clean":"2023-10-16T08:16:27.989573+0000","last_became_active":"2023-10-16T08:16:27.182005+0000","last_became_peered":"2023-10-16T08:16:27.182005+0000","last_unstale":"2023-10-16T08:16:27.989573+0000","last_undegraded":"2023-10-16T08:16:27.989573+0000","last_fullsized":"2023-10-16T08:16:27.989573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:06.143173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985684+0000","last_change":"2023-10-16T08:16:21.945969+0000","last_active":"2023-10-16T08:16:26.985684+0000","last_peered":"2023-10-16T08:16:26.985684+0000","last_clean":"2023-10-16T08:16:26.985684+0000","last_became_active":"2023-10-16T08:16:21.945681+0000","last_became_peered":"2023-10-16T08:16:21.945681+0000","last_unstale":"2023-10-16T08:16:26.985684+0000","last_undegraded":"2023-10-16T08:16:26.985684+0000","last_fullsized":"2023-10-16T08:16:26.985684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988041+0000","last_change":"2023-10-16T08:16:22.974743+0000","last_active":"2023-10-16T08:16:26.988041+0000","last_peered":"2023-10-16T08:16:26.988041+0000","last_clean":"2023-10-16T08:16:26.988041+0000","last_became_active":"2023-10-16T08:16:22.974180+0000","last_became_peered":"2023-10-16T08:16:22.974180+0000","last_unstale":"2023-10-16T08:16:26.988041+0000","last_undegraded":"2023-10-16T08:16:26.988041+0000","last_fullsized":"2023-10-16T08:16:26.988041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542410+0000","last_change":"2023-10-16T08:16:27.011704+0000","last_active":"2023-10-16T08:16:29.542410+0000","last_peered":"2023-10-16T08:16:29.542410+0000","last_clean":"2023-10-16T08:16:29.542410+0000","last_became_active":"2023-10-16T08:16:27.011142+0000","last_became_peered":"2023-10-16T08:16:27.011142+0000","last_unstale":"2023-10-16T08:16:29.542410+0000","last_undegraded":"2023-10-16T08:16:29.542410+0000","last_fullsized":"2023-10-16T08:16:29.542410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:17.770482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178836+0000","last_change":"2023-10-16T08:16:21.964716+0000","last_active":"2023-10-16T08:16:27.178836+0000","last_peered":"2023-10-16T08:16:27.178836+0000","last_clean":"2023-10-16T08:16:27.178836+0000","last_became_active":"2023-10-16T08:16:21.964451+0000","last_became_peered":"2023-10-16T08:16:21.964451+0000","last_unstale":"2023-10-16T08:16:27.178836+0000","last_undegraded":"2023-10-16T08:16:27.178836+0000","last_fullsized":"2023-10-16T08:16:27.178836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994868+0000","last_change":"2023-10-16T08:16:22.971839+0000","last_active":"2023-10-16T08:16:26.994868+0000","last_peered":"2023-10-16T08:16:26.994868+0000","last_clean":"2023-10-16T08:16:26.994868+0000","last_became_active":"2023-10-16T08:16:22.971456+0000","last_became_peered":"2023-10-16T08:16:22.971456+0000","last_unstale":"2023-10-16T08:16:26.994868+0000","last_undegraded":"2023-10-16T08:16:26.994868+0000","last_fullsized":"2023-10-16T08:16:26.994868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543415+0000","last_change":"2023-10-16T08:16:27.024297+0000","last_active":"2023-10-16T08:16:29.543415+0000","last_peered":"2023-10-16T08:16:29.543415+0000","last_clean":"2023-10-16T08:16:29.543415+0000","last_became_active":"2023-10-16T08:16:27.023792+0000","last_became_peered":"2023-10-16T08:16:27.023792+0000","last_unstale":"2023-10-16T08:16:29.543415+0000","last_undegraded":"2023-10-16T08:16:29.543415+0000","last_fullsized":"2023-10-16T08:16:29.543415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:20.519605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178617+0000","last_change":"2023-10-16T08:16:21.964025+0000","last_active":"2023-10-16T08:16:27.178617+0000","last_peered":"2023-10-16T08:16:27.178617+0000","last_clean":"2023-10-16T08:16:27.178617+0000","last_became_active":"2023-10-16T08:16:21.963582+0000","last_became_peered":"2023-10-16T08:16:21.963582+0000","last_unstale":"2023-10-16T08:16:27.178617+0000","last_undegraded":"2023-10-16T08:16:27.178617+0000","last_fullsized":"2023-10-16T08:16:27.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994775+0000","last_change":"2023-10-16T08:16:22.980735+0000","last_active":"2023-10-16T08:16:26.994775+0000","last_peered":"2023-10-16T08:16:26.994775+0000","last_clean":"2023-10-16T08:16:26.994775+0000","last_became_active":"2023-10-16T08:16:22.980409+0000","last_became_peered":"2023-10-16T08:16:22.980409+0000","last_unstale":"2023-10-16T08:16:26.994775+0000","last_undegraded":"2023-10-16T08:16:26.994775+0000","last_fullsized":"2023-10-16T08:16:26.994775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542893+0000","last_change":"2023-10-16T08:16:27.024269+0000","last_active":"2023-10-16T08:16:29.542893+0000","last_peered":"2023-10-16T08:16:29.542893+0000","last_clean":"2023-10-16T08:16:29.542893+0000","last_became_active":"2023-10-16T08:16:27.023822+0000","last_became_peered":"2023-10-16T08:16:27.023822+0000","last_unstale":"2023-10-16T08:16:29.542893+0000","last_undegraded":"2023-10-16T08:16:29.542893+0000","last_fullsized":"2023-10-16T08:16:29.542893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:49.956062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987114+0000","last_change":"2023-10-16T08:16:22.969401+0000","last_active":"2023-10-16T08:16:26.987114+0000","last_peered":"2023-10-16T08:16:26.987114+0000","last_clean":"2023-10-16T08:16:26.987114+0000","last_became_active":"2023-10-16T08:16:22.969099+0000","last_became_peered":"2023-10-16T08:16:22.969099+0000","last_unstale":"2023-10-16T08:16:26.987114+0000","last_undegraded":"2023-10-16T08:16:26.987114+0000","last_fullsized":"2023-10-16T08:16:26.987114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178720+0000","last_change":"2023-10-16T08:16:21.961388+0000","last_active":"2023-10-16T08:16:27.178720+0000","last_peered":"2023-10-16T08:16:27.178720+0000","last_clean":"2023-10-16T08:16:27.178720+0000","last_became_active":"2023-10-16T08:16:21.961050+0000","last_became_peered":"2023-10-16T08:16:21.961050+0000","last_unstale":"2023-10-16T08:16:27.178720+0000","last_undegraded":"2023-10-16T08:16:27.178720+0000","last_fullsized":"2023-10-16T08:16:27.178720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:13.387501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543302+0000","last_change":"2023-10-16T08:16:27.019072+0000","last_active":"2023-10-16T08:16:29.543302+0000","last_peered":"2023-10-16T08:16:29.543302+0000","last_clean":"2023-10-16T08:16:29.543302+0000","last_became_active":"2023-10-16T08:16:27.018440+0000","last_became_peered":"2023-10-16T08:16:27.018440+0000","last_unstale":"2023-10-16T08:16:29.543302+0000","last_undegraded":"2023-10-16T08:16:29.543302+0000","last_fullsized":"2023-10-16T08:16:29.543302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:39.310251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990373+0000","last_change":"2023-10-16T08:16:22.960667+0000","last_active":"2023-10-16T08:16:26.990373+0000","last_peered":"2023-10-16T08:16:26.990373+0000","last_clean":"2023-10-16T08:16:26.990373+0000","last_became_active":"2023-10-16T08:16:22.960345+0000","last_became_peered":"2023-10-16T08:16:22.960345+0000","last_unstale":"2023-10-16T08:16:26.990373+0000","last_undegraded":"2023-10-16T08:16:26.990373+0000","last_fullsized":"2023-10-16T08:16:26.990373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013495+0000","last_change":"2023-10-16T08:16:21.950408+0000","last_active":"2023-10-16T08:16:27.013495+0000","last_peered":"2023-10-16T08:16:27.013495+0000","last_clean":"2023-10-16T08:16:27.013495+0000","last_became_active":"2023-10-16T08:16:21.947942+0000","last_became_peered":"2023-10-16T08:16:21.947942+0000","last_unstale":"2023-10-16T08:16:27.013495+0000","last_undegraded":"2023-10-16T08:16:27.013495+0000","last_fullsized":"2023-10-16T08:16:27.013495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542836+0000","last_change":"2023-10-16T08:16:27.021686+0000","last_active":"2023-10-16T08:16:29.542836+0000","last_peered":"2023-10-16T08:16:29.542836+0000","last_clean":"2023-10-16T08:16:29.542836+0000","last_became_active":"2023-10-16T08:16:27.021230+0000","last_became_peered":"2023-10-16T08:16:27.021230+0000","last_unstale":"2023-10-16T08:16:29.542836+0000","last_undegraded":"2023-10-16T08:16:29.542836+0000","last_fullsized":"2023-10-16T08:16:29.542836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:12.583792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986505+0000","last_change":"2023-10-16T08:16:22.955965+0000","last_active":"2023-10-16T08:16:26.986505+0000","last_peered":"2023-10-16T08:16:26.986505+0000","last_clean":"2023-10-16T08:16:26.986505+0000","last_became_active":"2023-10-16T08:16:22.955652+0000","last_became_peered":"2023-10-16T08:16:22.955652+0000","last_unstale":"2023-10-16T08:16:26.986505+0000","last_undegraded":"2023-10-16T08:16:26.986505+0000","last_fullsized":"2023-10-16T08:16:26.986505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009595+0000","last_change":"2023-10-16T08:16:21.950219+0000","last_active":"2023-10-16T08:16:27.009595+0000","last_peered":"2023-10-16T08:16:27.009595+0000","last_clean":"2023-10-16T08:16:27.009595+0000","last_became_active":"2023-10-16T08:16:21.949749+0000","last_became_peered":"2023-10-16T08:16:21.949749+0000","last_unstale":"2023-10-16T08:16:27.009595+0000","last_undegraded":"2023-10-16T08:16:27.009595+0000","last_fullsized":"2023-10-16T08:16:27.009595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988141+0000","last_change":"2023-10-16T08:16:27.182341+0000","last_active":"2023-10-16T08:16:27.988141+0000","last_peered":"2023-10-16T08:16:27.988141+0000","last_clean":"2023-10-16T08:16:27.988141+0000","last_became_active":"2023-10-16T08:16:27.181854+0000","last_became_peered":"2023-10-16T08:16:27.181854+0000","last_unstale":"2023-10-16T08:16:27.988141+0000","last_undegraded":"2023-10-16T08:16:27.988141+0000","last_fullsized":"2023-10-16T08:16:27.988141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003825+0000","last_change":"2023-10-16T08:16:22.962471+0000","last_active":"2023-10-16T08:16:27.003825+0000","last_peered":"2023-10-16T08:16:27.003825+0000","last_clean":"2023-10-16T08:16:27.003825+0000","last_became_active":"2023-10-16T08:16:22.961915+0000","last_became_peered":"2023-10-16T08:16:22.961915+0000","last_unstale":"2023-10-16T08:16:27.003825+0000","last_undegraded":"2023-10-16T08:16:27.003825+0000","last_fullsized":"2023-10-16T08:16:27.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:13.023207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177873+0000","last_change":"2023-10-16T08:16:21.964429+0000","last_active":"2023-10-16T08:16:27.177873+0000","last_peered":"2023-10-16T08:16:27.177873+0000","last_clean":"2023-10-16T08:16:27.177873+0000","last_became_active":"2023-10-16T08:16:21.964154+0000","last_became_peered":"2023-10-16T08:16:21.964154+0000","last_unstale":"2023-10-16T08:16:27.177873+0000","last_undegraded":"2023-10-16T08:16:27.177873+0000","last_fullsized":"2023-10-16T08:16:27.177873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543629+0000","last_change":"2023-10-16T08:16:27.027359+0000","last_active":"2023-10-16T08:16:29.543629+0000","last_peered":"2023-10-16T08:16:29.543629+0000","last_clean":"2023-10-16T08:16:29.543629+0000","last_became_active":"2023-10-16T08:16:27.026923+0000","last_became_peered":"2023-10-16T08:16:27.026923+0000","last_unstale":"2023-10-16T08:16:29.543629+0000","last_undegraded":"2023-10-16T08:16:29.543629+0000","last_fullsized":"2023-10-16T08:16:29.543629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.989245+0000","last_change":"2023-10-16T08:16:22.967177+0000","last_active":"2023-10-16T08:16:26.989245+0000","last_peered":"2023-10-16T08:16:26.989245+0000","last_clean":"2023-10-16T08:16:26.989245+0000","last_became_active":"2023-10-16T08:16:22.966812+0000","last_became_peered":"2023-10-16T08:16:22.966812+0000","last_unstale":"2023-10-16T08:16:26.989245+0000","last_undegraded":"2023-10-16T08:16:26.989245+0000","last_fullsized":"2023-10-16T08:16:26.989245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988555+0000","last_change":"2023-10-16T08:16:21.958338+0000","last_active":"2023-10-16T08:16:26.988555+0000","last_peered":"2023-10-16T08:16:26.988555+0000","last_clean":"2023-10-16T08:16:26.988555+0000","last_became_active":"2023-10-16T08:16:21.958052+0000","last_became_peered":"2023-10-16T08:16:21.958052+0000","last_unstale":"2023-10-16T08:16:26.988555+0000","last_undegraded":"2023-10-16T08:16:26.988555+0000","last_fullsized":"2023-10-16T08:16:26.988555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543530+0000","last_change":"2023-10-16T08:16:27.184138+0000","last_active":"2023-10-16T08:16:29.543530+0000","last_peered":"2023-10-16T08:16:29.543530+0000","last_clean":"2023-10-16T08:16:29.543530+0000","last_became_active":"2023-10-16T08:16:27.183769+0000","last_became_peered":"2023-10-16T08:16:27.183769+0000","last_unstale":"2023-10-16T08:16:29.543530+0000","last_undegraded":"2023-10-16T08:16:29.543530+0000","last_fullsized":"2023-10-16T08:16:29.543530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987117+0000","last_change":"2023-10-16T08:16:22.973708+0000","last_active":"2023-10-16T08:16:26.987117+0000","last_peered":"2023-10-16T08:16:26.987117+0000","last_clean":"2023-10-16T08:16:26.987117+0000","last_became_active":"2023-10-16T08:16:22.973402+0000","last_became_peered":"2023-10-16T08:16:22.973402+0000","last_unstale":"2023-10-16T08:16:26.987117+0000","last_undegraded":"2023-10-16T08:16:26.987117+0000","last_fullsized":"2023-10-16T08:16:26.987117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014342+0000","last_change":"2023-10-16T08:16:21.955020+0000","last_active":"2023-10-16T08:16:27.014342+0000","last_peered":"2023-10-16T08:16:27.014342+0000","last_clean":"2023-10-16T08:16:27.014342+0000","last_became_active":"2023-10-16T08:16:21.954681+0000","last_became_peered":"2023-10-16T08:16:21.954681+0000","last_unstale":"2023-10-16T08:16:27.014342+0000","last_undegraded":"2023-10-16T08:16:27.014342+0000","last_fullsized":"2023-10-16T08:16:27.014342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989486+0000","last_change":"2023-10-16T08:16:27.179578+0000","last_active":"2023-10-16T08:16:27.989486+0000","last_peered":"2023-10-16T08:16:27.989486+0000","last_clean":"2023-10-16T08:16:27.989486+0000","last_became_active":"2023-10-16T08:16:27.178683+0000","last_became_peered":"2023-10-16T08:16:27.178683+0000","last_unstale":"2023-10-16T08:16:27.989486+0000","last_undegraded":"2023-10-16T08:16:27.989486+0000","last_fullsized":"2023-10-16T08:16:27.989486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014216+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.014216+0000","last_peered":"2023-10-16T08:16:27.014216+0000","last_clean":"2023-10-16T08:16:27.014216+0000","last_became_active":"2023-10-16T08:16:22.952824+0000","last_became_peered":"2023-10-16T08:16:22.952824+0000","last_unstale":"2023-10-16T08:16:27.014216+0000","last_undegraded":"2023-10-16T08:16:27.014216+0000","last_fullsized":"2023-10-16T08:16:27.014216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178889+0000","last_change":"2023-10-16T08:16:21.964665+0000","last_active":"2023-10-16T08:16:27.178889+0000","last_peered":"2023-10-16T08:16:27.178889+0000","last_clean":"2023-10-16T08:16:27.178889+0000","last_became_active":"2023-10-16T08:16:21.964105+0000","last_became_peered":"2023-10-16T08:16:21.964105+0000","last_unstale":"2023-10-16T08:16:27.178889+0000","last_undegraded":"2023-10-16T08:16:27.178889+0000","last_fullsized":"2023-10-16T08:16:27.178889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:12.709836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541103+0000","last_change":"2023-10-16T08:16:27.180634+0000","last_active":"2023-10-16T08:16:29.541103+0000","last_peered":"2023-10-16T08:16:29.541103+0000","last_clean":"2023-10-16T08:16:29.541103+0000","last_became_active":"2023-10-16T08:16:27.180186+0000","last_became_peered":"2023-10-16T08:16:27.180186+0000","last_unstale":"2023-10-16T08:16:29.541103+0000","last_undegraded":"2023-10-16T08:16:29.541103+0000","last_fullsized":"2023-10-16T08:16:29.541103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994880+0000","last_change":"2023-10-16T08:16:22.994702+0000","last_active":"2023-10-16T08:16:26.994880+0000","last_peered":"2023-10-16T08:16:26.994880+0000","last_clean":"2023-10-16T08:16:26.994880+0000","last_became_active":"2023-10-16T08:16:22.990441+0000","last_became_peered":"2023-10-16T08:16:22.990441+0000","last_unstale":"2023-10-16T08:16:26.994880+0000","last_undegraded":"2023-10-16T08:16:26.994880+0000","last_fullsized":"2023-10-16T08:16:26.994880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014301+0000","last_change":"2023-10-16T08:16:21.942771+0000","last_active":"2023-10-16T08:16:27.014301+0000","last_peered":"2023-10-16T08:16:27.014301+0000","last_clean":"2023-10-16T08:16:27.014301+0000","last_became_active":"2023-10-16T08:16:21.942441+0000","last_became_peered":"2023-10-16T08:16:21.942441+0000","last_unstale":"2023-10-16T08:16:27.014301+0000","last_undegraded":"2023-10-16T08:16:27.014301+0000","last_fullsized":"2023-10-16T08:16:27.014301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.986815+0000","last_change":"2023-10-16T08:16:27.013697+0000","last_active":"2023-10-16T08:16:27.986815+0000","last_peered":"2023-10-16T08:16:27.986815+0000","last_clean":"2023-10-16T08:16:27.986815+0000","last_became_active":"2023-10-16T08:16:27.012937+0000","last_became_peered":"2023-10-16T08:16:27.012937+0000","last_unstale":"2023-10-16T08:16:27.986815+0000","last_undegraded":"2023-10-16T08:16:27.986815+0000","last_fullsized":"2023-10-16T08:16:27.986815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003417+0000","last_change":"2023-10-16T08:16:22.964069+0000","last_active":"2023-10-16T08:16:27.003417+0000","last_peered":"2023-10-16T08:16:27.003417+0000","last_clean":"2023-10-16T08:16:27.003417+0000","last_became_active":"2023-10-16T08:16:22.963802+0000","last_became_peered":"2023-10-16T08:16:22.963802+0000","last_unstale":"2023-10-16T08:16:27.003417+0000","last_undegraded":"2023-10-16T08:16:27.003417+0000","last_fullsized":"2023-10-16T08:16:27.003417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994257+0000","last_change":"2023-10-16T08:16:21.962674+0000","last_active":"2023-10-16T08:16:26.994257+0000","last_peered":"2023-10-16T08:16:26.994257+0000","last_clean":"2023-10-16T08:16:26.994257+0000","last_became_active":"2023-10-16T08:16:21.962316+0000","last_became_peered":"2023-10-16T08:16:21.962316+0000","last_unstale":"2023-10-16T08:16:26.994257+0000","last_undegraded":"2023-10-16T08:16:26.994257+0000","last_fullsized":"2023-10-16T08:16:26.994257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:32.032856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987533+0000","last_change":"2023-10-16T08:16:27.020747+0000","last_active":"2023-10-16T08:16:27.987533+0000","last_peered":"2023-10-16T08:16:27.987533+0000","last_clean":"2023-10-16T08:16:27.987533+0000","last_became_active":"2023-10-16T08:16:27.019634+0000","last_became_peered":"2023-10-16T08:16:27.019634+0000","last_unstale":"2023-10-16T08:16:27.987533+0000","last_undegraded":"2023-10-16T08:16:27.987533+0000","last_fullsized":"2023-10-16T08:16:27.987533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009729+0000","last_change":"2023-10-16T08:16:22.953894+0000","last_active":"2023-10-16T08:16:27.009729+0000","last_peered":"2023-10-16T08:16:27.009729+0000","last_clean":"2023-10-16T08:16:27.009729+0000","last_became_active":"2023-10-16T08:16:22.953606+0000","last_became_peered":"2023-10-16T08:16:22.953606+0000","last_unstale":"2023-10-16T08:16:27.009729+0000","last_undegraded":"2023-10-16T08:16:27.009729+0000","last_fullsized":"2023-10-16T08:16:27.009729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996654+0000","last_change":"2023-10-16T08:16:21.946996+0000","last_active":"2023-10-16T08:16:26.996654+0000","last_peered":"2023-10-16T08:16:26.996654+0000","last_clean":"2023-10-16T08:16:26.996654+0000","last_became_active":"2023-10-16T08:16:21.946675+0000","last_became_peered":"2023-10-16T08:16:21.946675+0000","last_unstale":"2023-10-16T08:16:26.996654+0000","last_undegraded":"2023-10-16T08:16:26.996654+0000","last_fullsized":"2023-10-16T08:16:26.996654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:43:55.303584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988192+0000","last_change":"2023-10-16T08:16:27.020112+0000","last_active":"2023-10-16T08:16:27.988192+0000","last_peered":"2023-10-16T08:16:27.988192+0000","last_clean":"2023-10-16T08:16:27.988192+0000","last_became_active":"2023-10-16T08:16:27.019606+0000","last_became_peered":"2023-10-16T08:16:27.019606+0000","last_unstale":"2023-10-16T08:16:27.988192+0000","last_undegraded":"2023-10-16T08:16:27.988192+0000","last_fullsized":"2023-10-16T08:16:27.988192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984195+0000","last_change":"2023-10-16T08:16:22.962655+0000","last_active":"2023-10-16T08:16:26.984195+0000","last_peered":"2023-10-16T08:16:26.984195+0000","last_clean":"2023-10-16T08:16:26.984195+0000","last_became_active":"2023-10-16T08:16:22.962344+0000","last_became_peered":"2023-10-16T08:16:22.962344+0000","last_unstale":"2023-10-16T08:16:26.984195+0000","last_undegraded":"2023-10-16T08:16:26.984195+0000","last_fullsized":"2023-10-16T08:16:26.984195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:43.247033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177937+0000","last_change":"2023-10-16T08:16:21.959357+0000","last_active":"2023-10-16T08:16:27.177937+0000","last_peered":"2023-10-16T08:16:27.177937+0000","last_clean":"2023-10-16T08:16:27.177937+0000","last_became_active":"2023-10-16T08:16:21.959015+0000","last_became_peered":"2023-10-16T08:16:21.959015+0000","last_unstale":"2023-10-16T08:16:27.177937+0000","last_undegraded":"2023-10-16T08:16:27.177937+0000","last_fullsized":"2023-10-16T08:16:27.177937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:02.223178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541384+0000","last_change":"2023-10-16T08:16:27.179195+0000","last_active":"2023-10-16T08:16:29.541384+0000","last_peered":"2023-10-16T08:16:29.541384+0000","last_clean":"2023-10-16T08:16:29.541384+0000","last_became_active":"2023-10-16T08:16:27.178732+0000","last_became_peered":"2023-10-16T08:16:27.178732+0000","last_unstale":"2023-10-16T08:16:29.541384+0000","last_undegraded":"2023-10-16T08:16:29.541384+0000","last_fullsized":"2023-10-16T08:16:29.541384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986752+0000","last_change":"2023-10-16T08:16:22.973605+0000","last_active":"2023-10-16T08:16:26.986752+0000","last_peered":"2023-10-16T08:16:26.986752+0000","last_clean":"2023-10-16T08:16:26.986752+0000","last_became_active":"2023-10-16T08:16:22.973281+0000","last_became_peered":"2023-10-16T08:16:22.973281+0000","last_unstale":"2023-10-16T08:16:26.986752+0000","last_undegraded":"2023-10-16T08:16:26.986752+0000","last_fullsized":"2023-10-16T08:16:26.986752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994407+0000","last_change":"2023-10-16T08:16:21.950850+0000","last_active":"2023-10-16T08:16:26.994407+0000","last_peered":"2023-10-16T08:16:26.994407+0000","last_clean":"2023-10-16T08:16:26.994407+0000","last_became_active":"2023-10-16T08:16:21.950526+0000","last_became_peered":"2023-10-16T08:16:21.950526+0000","last_unstale":"2023-10-16T08:16:26.994407+0000","last_undegraded":"2023-10-16T08:16:26.994407+0000","last_fullsized":"2023-10-16T08:16:26.994407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177966+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:16:27.177966+0000","last_peered":"2023-10-16T08:16:27.177966+0000","last_clean":"2023-10-16T08:16:27.177966+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:16:27.177966+0000","last_undegraded":"2023-10-16T08:16:27.177966+0000","last_fullsized":"2023-10-16T08:16:27.177966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991969+0000","last_change":"2023-10-16T08:16:27.184250+0000","last_active":"2023-10-16T08:16:27.991969+0000","last_peered":"2023-10-16T08:16:27.991969+0000","last_clean":"2023-10-16T08:16:27.991969+0000","last_became_active":"2023-10-16T08:16:27.183826+0000","last_became_peered":"2023-10-16T08:16:27.183826+0000","last_unstale":"2023-10-16T08:16:27.991969+0000","last_undegraded":"2023-10-16T08:16:27.991969+0000","last_fullsized":"2023-10-16T08:16:27.991969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178713+0000","last_change":"2023-10-16T08:16:22.975303+0000","last_active":"2023-10-16T08:16:27.178713+0000","last_peered":"2023-10-16T08:16:27.178713+0000","last_clean":"2023-10-16T08:16:27.178713+0000","last_became_active":"2023-10-16T08:16:22.974890+0000","last_became_peered":"2023-10-16T08:16:22.974890+0000","last_unstale":"2023-10-16T08:16:27.178713+0000","last_undegraded":"2023-10-16T08:16:27.178713+0000","last_fullsized":"2023-10-16T08:16:27.178713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.997283+0000","last_change":"2023-10-16T08:16:21.959337+0000","last_active":"2023-10-16T08:16:26.997283+0000","last_peered":"2023-10-16T08:16:26.997283+0000","last_clean":"2023-10-16T08:16:26.997283+0000","last_became_active":"2023-10-16T08:16:21.959059+0000","last_became_peered":"2023-10-16T08:16:21.959059+0000","last_unstale":"2023-10-16T08:16:26.997283+0000","last_undegraded":"2023-10-16T08:16:26.997283+0000","last_fullsized":"2023-10-16T08:16:26.997283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991525+0000","last_change":"2023-10-16T08:16:27.181051+0000","last_active":"2023-10-16T08:16:27.991525+0000","last_peered":"2023-10-16T08:16:27.991525+0000","last_clean":"2023-10-16T08:16:27.991525+0000","last_became_active":"2023-10-16T08:16:27.180503+0000","last_became_peered":"2023-10-16T08:16:27.180503+0000","last_unstale":"2023-10-16T08:16:27.991525+0000","last_undegraded":"2023-10-16T08:16:27.991525+0000","last_fullsized":"2023-10-16T08:16:27.991525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994965+0000","last_change":"2023-10-16T08:16:22.990070+0000","last_active":"2023-10-16T08:16:26.994965+0000","last_peered":"2023-10-16T08:16:26.994965+0000","last_clean":"2023-10-16T08:16:26.994965+0000","last_became_active":"2023-10-16T08:16:22.989787+0000","last_became_peered":"2023-10-16T08:16:22.989787+0000","last_unstale":"2023-10-16T08:16:26.994965+0000","last_undegraded":"2023-10-16T08:16:26.994965+0000","last_fullsized":"2023-10-16T08:16:26.994965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013982+0000","last_change":"2023-10-16T08:16:21.961287+0000","last_active":"2023-10-16T08:16:27.013982+0000","last_peered":"2023-10-16T08:16:27.013982+0000","last_clean":"2023-10-16T08:16:27.013982+0000","last_became_active":"2023-10-16T08:16:21.956293+0000","last_became_peered":"2023-10-16T08:16:21.956293+0000","last_unstale":"2023-10-16T08:16:27.013982+0000","last_undegraded":"2023-10-16T08:16:27.013982+0000","last_fullsized":"2023-10-16T08:16:27.013982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541257+0000","last_change":"2023-10-16T08:16:27.013097+0000","last_active":"2023-10-16T08:16:29.541257+0000","last_peered":"2023-10-16T08:16:29.541257+0000","last_clean":"2023-10-16T08:16:29.541257+0000","last_became_active":"2023-10-16T08:16:27.012587+0000","last_became_peered":"2023-10-16T08:16:27.012587+0000","last_unstale":"2023-10-16T08:16:29.541257+0000","last_undegraded":"2023-10-16T08:16:29.541257+0000","last_fullsized":"2023-10-16T08:16:29.541257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179067+0000","last_change":"2023-10-16T08:16:22.979584+0000","last_active":"2023-10-16T08:16:27.179067+0000","last_peered":"2023-10-16T08:16:27.179067+0000","last_clean":"2023-10-16T08:16:27.179067+0000","last_became_active":"2023-10-16T08:16:22.978239+0000","last_became_peered":"2023-10-16T08:16:22.978239+0000","last_unstale":"2023-10-16T08:16:27.179067+0000","last_undegraded":"2023-10-16T08:16:27.179067+0000","last_fullsized":"2023-10-16T08:16:27.179067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:27:57.200006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996247+0000","last_change":"2023-10-16T08:16:21.947037+0000","last_active":"2023-10-16T08:16:26.996247+0000","last_peered":"2023-10-16T08:16:26.996247+0000","last_clean":"2023-10-16T08:16:26.996247+0000","last_became_active":"2023-10-16T08:16:21.946671+0000","last_became_peered":"2023-10-16T08:16:21.946671+0000","last_unstale":"2023-10-16T08:16:26.996247+0000","last_undegraded":"2023-10-16T08:16:26.996247+0000","last_fullsized":"2023-10-16T08:16:26.996247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542197+0000","last_change":"2023-10-16T08:16:27.189477+0000","last_active":"2023-10-16T08:16:29.542197+0000","last_peered":"2023-10-16T08:16:29.542197+0000","last_clean":"2023-10-16T08:16:29.542197+0000","last_became_active":"2023-10-16T08:16:27.185942+0000","last_became_peered":"2023-10-16T08:16:27.185942+0000","last_unstale":"2023-10-16T08:16:29.542197+0000","last_undegraded":"2023-10-16T08:16:29.542197+0000","last_fullsized":"2023-10-16T08:16:29.542197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:13.645047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.973600+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.973215+0000","last_became_peered":"2023-10-16T08:16:22.973215+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.005005+0000","last_change":"2023-10-16T08:16:21.949416+0000","last_active":"2023-10-16T08:16:27.005005+0000","last_peered":"2023-10-16T08:16:27.005005+0000","last_clean":"2023-10-16T08:16:27.005005+0000","last_became_active":"2023-10-16T08:16:21.948836+0000","last_became_peered":"2023-10-16T08:16:21.948836+0000","last_unstale":"2023-10-16T08:16:27.005005+0000","last_undegraded":"2023-10-16T08:16:27.005005+0000","last_fullsized":"2023-10-16T08:16:27.005005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543245+0000","last_change":"2023-10-16T08:16:27.012879+0000","last_active":"2023-10-16T08:16:29.543245+0000","last_peered":"2023-10-16T08:16:29.543245+0000","last_clean":"2023-10-16T08:16:29.543245+0000","last_became_active":"2023-10-16T08:16:27.012466+0000","last_became_peered":"2023-10-16T08:16:27.012466+0000","last_unstale":"2023-10-16T08:16:29.543245+0000","last_undegraded":"2023-10-16T08:16:29.543245+0000","last_fullsized":"2023-10-16T08:16:29.543245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:16.486175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178093+0000","last_change":"2023-10-16T08:16:22.973193+0000","last_active":"2023-10-16T08:16:27.178093+0000","last_peered":"2023-10-16T08:16:27.178093+0000","last_clean":"2023-10-16T08:16:27.178093+0000","last_became_active":"2023-10-16T08:16:22.972472+0000","last_became_peered":"2023-10-16T08:16:22.972472+0000","last_unstale":"2023-10-16T08:16:27.178093+0000","last_undegraded":"2023-10-16T08:16:27.178093+0000","last_fullsized":"2023-10-16T08:16:27.178093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996389+0000","last_change":"2023-10-16T08:16:21.952668+0000","last_active":"2023-10-16T08:16:26.996389+0000","last_peered":"2023-10-16T08:16:26.996389+0000","last_clean":"2023-10-16T08:16:26.996389+0000","last_became_active":"2023-10-16T08:16:21.952006+0000","last_became_peered":"2023-10-16T08:16:21.952006+0000","last_unstale":"2023-10-16T08:16:26.996389+0000","last_undegraded":"2023-10-16T08:16:26.996389+0000","last_fullsized":"2023-10-16T08:16:26.996389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541359+0000","last_change":"2023-10-16T08:16:27.015675+0000","last_active":"2023-10-16T08:16:29.541359+0000","last_peered":"2023-10-16T08:16:29.541359+0000","last_clean":"2023-10-16T08:16:29.541359+0000","last_became_active":"2023-10-16T08:16:27.015279+0000","last_became_peered":"2023-10-16T08:16:27.015279+0000","last_unstale":"2023-10-16T08:16:29.541359+0000","last_undegraded":"2023-10-16T08:16:29.541359+0000","last_fullsized":"2023-10-16T08:16:29.541359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986776+0000","last_change":"2023-10-16T08:16:22.963194+0000","last_active":"2023-10-16T08:16:26.986776+0000","last_peered":"2023-10-16T08:16:26.986776+0000","last_clean":"2023-10-16T08:16:26.986776+0000","last_became_active":"2023-10-16T08:16:22.962589+0000","last_became_peered":"2023-10-16T08:16:22.962589+0000","last_unstale":"2023-10-16T08:16:26.986776+0000","last_undegraded":"2023-10-16T08:16:26.986776+0000","last_fullsized":"2023-10-16T08:16:26.986776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986872+0000","last_change":"2023-10-16T08:16:21.957192+0000","last_active":"2023-10-16T08:16:26.986872+0000","last_peered":"2023-10-16T08:16:26.986872+0000","last_clean":"2023-10-16T08:16:26.986872+0000","last_became_active":"2023-10-16T08:16:21.955564+0000","last_became_peered":"2023-10-16T08:16:21.955564+0000","last_unstale":"2023-10-16T08:16:26.986872+0000","last_undegraded":"2023-10-16T08:16:26.986872+0000","last_fullsized":"2023-10-16T08:16:26.986872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:58.308103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990800+0000","last_change":"2023-10-16T08:16:27.185040+0000","last_active":"2023-10-16T08:16:27.990800+0000","last_peered":"2023-10-16T08:16:27.990800+0000","last_clean":"2023-10-16T08:16:27.990800+0000","last_became_active":"2023-10-16T08:16:27.184448+0000","last_became_peered":"2023-10-16T08:16:27.184448+0000","last_unstale":"2023-10-16T08:16:27.990800+0000","last_undegraded":"2023-10-16T08:16:27.990800+0000","last_fullsized":"2023-10-16T08:16:27.990800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:59.852030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178532+0000","last_change":"2023-10-16T08:16:22.980031+0000","last_active":"2023-10-16T08:16:27.178532+0000","last_peered":"2023-10-16T08:16:27.178532+0000","last_clean":"2023-10-16T08:16:27.178532+0000","last_became_active":"2023-10-16T08:16:22.976708+0000","last_became_peered":"2023-10-16T08:16:22.976708+0000","last_unstale":"2023-10-16T08:16:27.178532+0000","last_undegraded":"2023-10-16T08:16:27.178532+0000","last_fullsized":"2023-10-16T08:16:27.178532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994242+0000","last_change":"2023-10-16T08:16:21.956986+0000","last_active":"2023-10-16T08:16:26.994242+0000","last_peered":"2023-10-16T08:16:26.994242+0000","last_clean":"2023-10-16T08:16:26.994242+0000","last_became_active":"2023-10-16T08:16:21.956307+0000","last_became_peered":"2023-10-16T08:16:21.956307+0000","last_unstale":"2023-10-16T08:16:26.994242+0000","last_undegraded":"2023-10-16T08:16:26.994242+0000","last_fullsized":"2023-10-16T08:16:26.994242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:26.142575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990801+0000","last_change":"2023-10-16T08:16:27.179150+0000","last_active":"2023-10-16T08:16:27.990801+0000","last_peered":"2023-10-16T08:16:27.990801+0000","last_clean":"2023-10-16T08:16:27.990801+0000","last_became_active":"2023-10-16T08:16:27.178649+0000","last_became_peered":"2023-10-16T08:16:27.178649+0000","last_unstale":"2023-10-16T08:16:27.990801+0000","last_undegraded":"2023-10-16T08:16:27.990801+0000","last_fullsized":"2023-10-16T08:16:27.990801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:45.567047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984308+0000","last_change":"2023-10-16T08:16:22.967505+0000","last_active":"2023-10-16T08:16:26.984308+0000","last_peered":"2023-10-16T08:16:26.984308+0000","last_clean":"2023-10-16T08:16:26.984308+0000","last_became_active":"2023-10-16T08:16:22.967214+0000","last_became_peered":"2023-10-16T08:16:22.967214+0000","last_unstale":"2023-10-16T08:16:26.984308+0000","last_undegraded":"2023-10-16T08:16:26.984308+0000","last_fullsized":"2023-10-16T08:16:26.984308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988187+0000","last_change":"2023-10-16T08:16:21.946248+0000","last_active":"2023-10-16T08:16:26.988187+0000","last_peered":"2023-10-16T08:16:26.988187+0000","last_clean":"2023-10-16T08:16:26.988187+0000","last_became_active":"2023-10-16T08:16:21.945958+0000","last_became_peered":"2023-10-16T08:16:21.945958+0000","last_unstale":"2023-10-16T08:16:26.988187+0000","last_undegraded":"2023-10-16T08:16:26.988187+0000","last_fullsized":"2023-10-16T08:16:26.988187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543632+0000","last_change":"2023-10-16T08:16:27.184504+0000","last_active":"2023-10-16T08:16:29.543632+0000","last_peered":"2023-10-16T08:16:29.543632+0000","last_clean":"2023-10-16T08:16:29.543632+0000","last_became_active":"2023-10-16T08:16:27.184029+0000","last_became_peered":"2023-10-16T08:16:27.184029+0000","last_unstale":"2023-10-16T08:16:29.543632+0000","last_undegraded":"2023-10-16T08:16:29.543632+0000","last_fullsized":"2023-10-16T08:16:29.543632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178733+0000","last_change":"2023-10-16T08:16:22.986066+0000","last_active":"2023-10-16T08:16:27.178733+0000","last_peered":"2023-10-16T08:16:27.178733+0000","last_clean":"2023-10-16T08:16:27.178733+0000","last_became_active":"2023-10-16T08:16:22.985791+0000","last_became_peered":"2023-10-16T08:16:22.985791+0000","last_unstale":"2023-10-16T08:16:27.178733+0000","last_undegraded":"2023-10-16T08:16:27.178733+0000","last_fullsized":"2023-10-16T08:16:27.178733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:38.000149+0000","last_change":"2023-10-16T08:16:21.955781+0000","last_active":"2023-10-16T08:16:38.000149+0000","last_peered":"2023-10-16T08:16:38.000149+0000","last_clean":"2023-10-16T08:16:38.000149+0000","last_became_active":"2023-10-16T08:16:21.955327+0000","last_became_peered":"2023-10-16T08:16:21.955327+0000","last_unstale":"2023-10-16T08:16:38.000149+0000","last_undegraded":"2023-10-16T08:16:38.000149+0000","last_fullsized":"2023-10-16T08:16:38.000149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987157+0000","last_change":"2023-10-16T08:16:27.007137+0000","last_active":"2023-10-16T08:16:27.987157+0000","last_peered":"2023-10-16T08:16:27.987157+0000","last_clean":"2023-10-16T08:16:27.987157+0000","last_became_active":"2023-10-16T08:16:27.006722+0000","last_became_peered":"2023-10-16T08:16:27.006722+0000","last_unstale":"2023-10-16T08:16:27.987157+0000","last_undegraded":"2023-10-16T08:16:27.987157+0000","last_fullsized":"2023-10-16T08:16:27.987157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004799+0000","last_change":"2023-10-16T08:16:22.976159+0000","last_active":"2023-10-16T08:16:27.004799+0000","last_peered":"2023-10-16T08:16:27.004799+0000","last_clean":"2023-10-16T08:16:27.004799+0000","last_became_active":"2023-10-16T08:16:22.975879+0000","last_became_peered":"2023-10-16T08:16:22.975879+0000","last_unstale":"2023-10-16T08:16:27.004799+0000","last_undegraded":"2023-10-16T08:16:27.004799+0000","last_fullsized":"2023-10-16T08:16:27.004799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:04.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013901+0000","last_change":"2023-10-16T08:16:21.961500+0000","last_active":"2023-10-16T08:16:27.013901+0000","last_peered":"2023-10-16T08:16:27.013901+0000","last_clean":"2023-10-16T08:16:27.013901+0000","last_became_active":"2023-10-16T08:16:21.959870+0000","last_became_peered":"2023-10-16T08:16:21.959870+0000","last_unstale":"2023-10-16T08:16:27.013901+0000","last_undegraded":"2023-10-16T08:16:27.013901+0000","last_fullsized":"2023-10-16T08:16:27.013901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543647+0000","last_change":"2023-10-16T08:16:27.018940+0000","last_active":"2023-10-16T08:16:29.543647+0000","last_peered":"2023-10-16T08:16:29.543647+0000","last_clean":"2023-10-16T08:16:29.543647+0000","last_became_active":"2023-10-16T08:16:27.018428+0000","last_became_peered":"2023-10-16T08:16:27.018428+0000","last_unstale":"2023-10-16T08:16:29.543647+0000","last_undegraded":"2023-10-16T08:16:29.543647+0000","last_fullsized":"2023-10-16T08:16:29.543647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988140+0000","last_change":"2023-10-16T08:16:22.974536+0000","last_active":"2023-10-16T08:16:26.988140+0000","last_peered":"2023-10-16T08:16:26.988140+0000","last_clean":"2023-10-16T08:16:26.988140+0000","last_became_active":"2023-10-16T08:16:22.973582+0000","last_became_peered":"2023-10-16T08:16:22.973582+0000","last_unstale":"2023-10-16T08:16:26.988140+0000","last_undegraded":"2023-10-16T08:16:26.988140+0000","last_fullsized":"2023-10-16T08:16:26.988140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995774+0000","last_change":"2023-10-16T08:16:21.960308+0000","last_active":"2023-10-16T08:16:26.995774+0000","last_peered":"2023-10-16T08:16:26.995774+0000","last_clean":"2023-10-16T08:16:26.995774+0000","last_became_active":"2023-10-16T08:16:21.959328+0000","last_became_peered":"2023-10-16T08:16:21.959328+0000","last_unstale":"2023-10-16T08:16:26.995774+0000","last_undegraded":"2023-10-16T08:16:26.995774+0000","last_fullsized":"2023-10-16T08:16:26.995774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:15.899771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989677+0000","last_change":"2023-10-16T08:16:27.007716+0000","last_active":"2023-10-16T08:16:27.989677+0000","last_peered":"2023-10-16T08:16:27.989677+0000","last_clean":"2023-10-16T08:16:27.989677+0000","last_became_active":"2023-10-16T08:16:27.006885+0000","last_became_peered":"2023-10-16T08:16:27.006885+0000","last_unstale":"2023-10-16T08:16:27.989677+0000","last_undegraded":"2023-10-16T08:16:27.989677+0000","last_fullsized":"2023-10-16T08:16:27.989677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:18.625895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995685+0000","last_change":"2023-10-16T08:16:22.990790+0000","last_active":"2023-10-16T08:16:26.995685+0000","last_peered":"2023-10-16T08:16:26.995685+0000","last_clean":"2023-10-16T08:16:26.995685+0000","last_became_active":"2023-10-16T08:16:22.987169+0000","last_became_peered":"2023-10-16T08:16:22.987169+0000","last_unstale":"2023-10-16T08:16:26.995685+0000","last_undegraded":"2023-10-16T08:16:26.995685+0000","last_fullsized":"2023-10-16T08:16:26.995685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988440+0000","last_change":"2023-10-16T08:16:21.956681+0000","last_active":"2023-10-16T08:16:26.988440+0000","last_peered":"2023-10-16T08:16:26.988440+0000","last_clean":"2023-10-16T08:16:26.988440+0000","last_became_active":"2023-10-16T08:16:21.956318+0000","last_became_peered":"2023-10-16T08:16:21.956318+0000","last_unstale":"2023-10-16T08:16:26.988440+0000","last_undegraded":"2023-10-16T08:16:26.988440+0000","last_fullsized":"2023-10-16T08:16:26.988440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989667+0000","last_change":"2023-10-16T08:16:27.181126+0000","last_active":"2023-10-16T08:16:27.989667+0000","last_peered":"2023-10-16T08:16:27.989667+0000","last_clean":"2023-10-16T08:16:27.989667+0000","last_became_active":"2023-10-16T08:16:27.180722+0000","last_became_peered":"2023-10-16T08:16:27.180722+0000","last_unstale":"2023-10-16T08:16:27.989667+0000","last_undegraded":"2023-10-16T08:16:27.989667+0000","last_fullsized":"2023-10-16T08:16:27.989667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:39:03.047765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541176+0000","last_change":"2023-10-16T08:16:27.183947+0000","last_active":"2023-10-16T08:16:29.541176+0000","last_peered":"2023-10-16T08:16:29.541176+0000","last_clean":"2023-10-16T08:16:29.541176+0000","last_became_active":"2023-10-16T08:16:27.183487+0000","last_became_peered":"2023-10-16T08:16:27.183487+0000","last_unstale":"2023-10-16T08:16:29.541176+0000","last_undegraded":"2023-10-16T08:16:29.541176+0000","last_fullsized":"2023-10-16T08:16:29.541176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009520+0000","last_change":"2023-10-16T08:16:22.958791+0000","last_active":"2023-10-16T08:16:27.009520+0000","last_peered":"2023-10-16T08:16:27.009520+0000","last_clean":"2023-10-16T08:16:27.009520+0000","last_became_active":"2023-10-16T08:16:22.958513+0000","last_became_peered":"2023-10-16T08:16:22.958513+0000","last_unstale":"2023-10-16T08:16:27.009520+0000","last_undegraded":"2023-10-16T08:16:27.009520+0000","last_fullsized":"2023-10-16T08:16:27.009520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994423+0000","last_change":"2023-10-16T08:16:21.954384+0000","last_active":"2023-10-16T08:16:26.994423+0000","last_peered":"2023-10-16T08:16:26.994423+0000","last_clean":"2023-10-16T08:16:26.994423+0000","last_became_active":"2023-10-16T08:16:21.953995+0000","last_became_peered":"2023-10-16T08:16:21.953995+0000","last_unstale":"2023-10-16T08:16:26.994423+0000","last_undegraded":"2023-10-16T08:16:26.994423+0000","last_fullsized":"2023-10-16T08:16:26.994423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:34.686144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.992231+0000","last_change":"2023-10-16T08:16:27.008716+0000","last_active":"2023-10-16T08:16:27.992231+0000","last_peered":"2023-10-16T08:16:27.992231+0000","last_clean":"2023-10-16T08:16:27.992231+0000","last_became_active":"2023-10-16T08:16:27.007942+0000","last_became_peered":"2023-10-16T08:16:27.007942+0000","last_unstale":"2023-10-16T08:16:27.992231+0000","last_undegraded":"2023-10-16T08:16:27.992231+0000","last_fullsized":"2023-10-16T08:16:27.992231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986408+0000","last_change":"2023-10-16T08:16:22.969567+0000","last_active":"2023-10-16T08:16:26.986408+0000","last_peered":"2023-10-16T08:16:26.986408+0000","last_clean":"2023-10-16T08:16:26.986408+0000","last_became_active":"2023-10-16T08:16:22.969172+0000","last_became_peered":"2023-10-16T08:16:22.969172+0000","last_unstale":"2023-10-16T08:16:26.986408+0000","last_undegraded":"2023-10-16T08:16:26.986408+0000","last_fullsized":"2023-10-16T08:16:26.986408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986465+0000","last_change":"2023-10-16T08:16:21.956899+0000","last_active":"2023-10-16T08:16:26.986465+0000","last_peered":"2023-10-16T08:16:26.986465+0000","last_clean":"2023-10-16T08:16:26.986465+0000","last_became_active":"2023-10-16T08:16:21.956446+0000","last_became_peered":"2023-10-16T08:16:21.956446+0000","last_unstale":"2023-10-16T08:16:26.986465+0000","last_undegraded":"2023-10-16T08:16:26.986465+0000","last_fullsized":"2023-10-16T08:16:26.986465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987260+0000","last_change":"2023-10-16T08:16:21.955151+0000","last_active":"2023-10-16T08:16:26.987260+0000","last_peered":"2023-10-16T08:16:26.987260+0000","last_clean":"2023-10-16T08:16:26.987260+0000","last_became_active":"2023-10-16T08:16:21.953524+0000","last_became_peered":"2023-10-16T08:16:21.953524+0000","last_unstale":"2023-10-16T08:16:26.987260+0000","last_undegraded":"2023-10-16T08:16:26.987260+0000","last_fullsized":"2023-10-16T08:16:26.987260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988088+0000","last_change":"2023-10-16T08:16:22.965640+0000","last_active":"2023-10-16T08:16:26.988088+0000","last_peered":"2023-10-16T08:16:26.988088+0000","last_clean":"2023-10-16T08:16:26.988088+0000","last_became_active":"2023-10-16T08:16:22.963142+0000","last_became_peered":"2023-10-16T08:16:22.963142+0000","last_unstale":"2023-10-16T08:16:26.988088+0000","last_undegraded":"2023-10-16T08:16:26.988088+0000","last_fullsized":"2023-10-16T08:16:26.988088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.540964+0000","last_change":"2023-10-16T08:16:27.020258+0000","last_active":"2023-10-16T08:16:29.540964+0000","last_peered":"2023-10-16T08:16:29.540964+0000","last_clean":"2023-10-16T08:16:29.540964+0000","last_became_active":"2023-10-16T08:16:27.019846+0000","last_became_peered":"2023-10-16T08:16:27.019846+0000","last_unstale":"2023-10-16T08:16:29.540964+0000","last_undegraded":"2023-10-16T08:16:29.540964+0000","last_fullsized":"2023-10-16T08:16:29.540964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:47.301298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990459+0000","last_change":"2023-10-16T08:16:21.959137+0000","last_active":"2023-10-16T08:16:26.990459+0000","last_peered":"2023-10-16T08:16:26.990459+0000","last_clean":"2023-10-16T08:16:26.990459+0000","last_became_active":"2023-10-16T08:16:21.958785+0000","last_became_peered":"2023-10-16T08:16:21.958785+0000","last_unstale":"2023-10-16T08:16:26.990459+0000","last_undegraded":"2023-10-16T08:16:26.990459+0000","last_fullsized":"2023-10-16T08:16:26.990459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.999013+0000","last_change":"2023-10-16T08:16:22.961411+0000","last_active":"2023-10-16T08:16:26.999013+0000","last_peered":"2023-10-16T08:16:26.999013+0000","last_clean":"2023-10-16T08:16:26.999013+0000","last_became_active":"2023-10-16T08:16:22.961004+0000","last_became_peered":"2023-10-16T08:16:22.961004+0000","last_unstale":"2023-10-16T08:16:26.999013+0000","last_undegraded":"2023-10-16T08:16:26.999013+0000","last_fullsized":"2023-10-16T08:16:26.999013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:11.319467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990247+0000","last_change":"2023-10-16T08:16:27.014631+0000","last_active":"2023-10-16T08:16:27.990247+0000","last_peered":"2023-10-16T08:16:27.990247+0000","last_clean":"2023-10-16T08:16:27.990247+0000","last_became_active":"2023-10-16T08:16:27.014246+0000","last_became_peered":"2023-10-16T08:16:27.014246+0000","last_unstale":"2023-10-16T08:16:27.990247+0000","last_undegraded":"2023-10-16T08:16:27.990247+0000","last_fullsized":"2023-10-16T08:16:27.990247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:43.540909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986773+0000","last_change":"2023-10-16T08:16:22.971903+0000","last_active":"2023-10-16T08:16:26.986773+0000","last_peered":"2023-10-16T08:16:26.986773+0000","last_clean":"2023-10-16T08:16:26.986773+0000","last_became_active":"2023-10-16T08:16:22.971522+0000","last_became_peered":"2023-10-16T08:16:22.971522+0000","last_unstale":"2023-10-16T08:16:26.986773+0000","last_undegraded":"2023-10-16T08:16:26.986773+0000","last_fullsized":"2023-10-16T08:16:26.986773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996570+0000","last_change":"2023-10-16T08:16:22.970072+0000","last_active":"2023-10-16T08:16:26.996570+0000","last_peered":"2023-10-16T08:16:26.996570+0000","last_clean":"2023-10-16T08:16:26.996570+0000","last_became_active":"2023-10-16T08:16:22.969751+0000","last_became_peered":"2023-10-16T08:16:22.969751+0000","last_unstale":"2023-10-16T08:16:26.996570+0000","last_undegraded":"2023-10-16T08:16:26.996570+0000","last_fullsized":"2023-10-16T08:16:26.996570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994134+0000","last_change":"2023-10-16T08:16:22.995816+0000","last_active":"2023-10-16T08:16:26.994134+0000","last_peered":"2023-10-16T08:16:26.994134+0000","last_clean":"2023-10-16T08:16:26.994134+0000","last_became_active":"2023-10-16T08:16:22.990930+0000","last_became_peered":"2023-10-16T08:16:22.990930+0000","last_unstale":"2023-10-16T08:16:26.994134+0000","last_undegraded":"2023-10-16T08:16:26.994134+0000","last_fullsized":"2023-10-16T08:16:26.994134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987998+0000","last_change":"2023-10-16T08:16:22.972789+0000","last_active":"2023-10-16T08:16:26.987998+0000","last_peered":"2023-10-16T08:16:26.987998+0000","last_clean":"2023-10-16T08:16:26.987998+0000","last_became_active":"2023-10-16T08:16:22.972473+0000","last_became_peered":"2023-10-16T08:16:22.972473+0000","last_unstale":"2023-10-16T08:16:26.987998+0000","last_undegraded":"2023-10-16T08:16:26.987998+0000","last_fullsized":"2023-10-16T08:16:26.987998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985911+0000","last_change":"2023-10-16T08:16:22.971086+0000","last_active":"2023-10-16T08:16:26.985911+0000","last_peered":"2023-10-16T08:16:26.985911+0000","last_clean":"2023-10-16T08:16:26.985911+0000","last_became_active":"2023-10-16T08:16:22.962675+0000","last_became_peered":"2023-10-16T08:16:22.962675+0000","last_unstale":"2023-10-16T08:16:26.985911+0000","last_undegraded":"2023-10-16T08:16:26.985911+0000","last_fullsized":"2023-10-16T08:16:26.985911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013810+0000","last_change":"2023-10-16T08:16:22.962278+0000","last_active":"2023-10-16T08:16:27.013810+0000","last_peered":"2023-10-16T08:16:27.013810+0000","last_clean":"2023-10-16T08:16:27.013810+0000","last_became_active":"2023-10-16T08:16:22.962006+0000","last_became_peered":"2023-10-16T08:16:22.962006+0000","last_unstale":"2023-10-16T08:16:27.013810+0000","last_undegraded":"2023-10-16T08:16:27.013810+0000","last_fullsized":"2023-10-16T08:16:27.013810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975064+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.974692+0000","last_became_peered":"2023-10-16T08:16:22.974692+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987037+0000","last_change":"2023-10-16T08:16:22.961194+0000","last_active":"2023-10-16T08:16:26.987037+0000","last_peered":"2023-10-16T08:16:26.987037+0000","last_clean":"2023-10-16T08:16:26.987037+0000","last_became_active":"2023-10-16T08:16:22.960884+0000","last_became_peered":"2023-10-16T08:16:22.960884+0000","last_unstale":"2023-10-16T08:16:26.987037+0000","last_undegraded":"2023-10-16T08:16:26.987037+0000","last_fullsized":"2023-10-16T08:16:26.987037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994288+0000","last_change":"2023-10-16T08:16:22.983850+0000","last_active":"2023-10-16T08:16:26.994288+0000","last_peered":"2023-10-16T08:16:26.994288+0000","last_clean":"2023-10-16T08:16:26.994288+0000","last_became_active":"2023-10-16T08:16:22.978686+0000","last_became_peered":"2023-10-16T08:16:22.978686+0000","last_unstale":"2023-10-16T08:16:26.994288+0000","last_undegraded":"2023-10-16T08:16:26.994288+0000","last_fullsized":"2023-10-16T08:16:26.994288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986300+0000","last_change":"2023-10-16T08:16:22.957405+0000","last_active":"2023-10-16T08:16:26.986300+0000","last_peered":"2023-10-16T08:16:26.986300+0000","last_clean":"2023-10-16T08:16:26.986300+0000","last_became_active":"2023-10-16T08:16:22.957105+0000","last_became_peered":"2023-10-16T08:16:22.957105+0000","last_unstale":"2023-10-16T08:16:26.986300+0000","last_undegraded":"2023-10-16T08:16:26.986300+0000","last_fullsized":"2023-10-16T08:16:26.986300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994045+0000","last_change":"2023-10-16T08:16:22.985035+0000","last_active":"2023-10-16T08:16:26.994045+0000","last_peered":"2023-10-16T08:16:26.994045+0000","last_clean":"2023-10-16T08:16:26.994045+0000","last_became_active":"2023-10-16T08:16:22.984750+0000","last_became_peered":"2023-10-16T08:16:22.984750+0000","last_unstale":"2023-10-16T08:16:26.994045+0000","last_undegraded":"2023-10-16T08:16:26.994045+0000","last_fullsized":"2023-10-16T08:16:26.994045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987250+0000","last_change":"2023-10-16T08:16:22.976322+0000","last_active":"2023-10-16T08:16:26.987250+0000","last_peered":"2023-10-16T08:16:26.987250+0000","last_clean":"2023-10-16T08:16:26.987250+0000","last_became_active":"2023-10-16T08:16:22.976064+0000","last_became_peered":"2023-10-16T08:16:22.976064+0000","last_unstale":"2023-10-16T08:16:26.987250+0000","last_undegraded":"2023-10-16T08:16:26.987250+0000","last_fullsized":"2023-10-16T08:16:26.987250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987111+0000","last_change":"2023-10-16T08:16:22.965222+0000","last_active":"2023-10-16T08:16:26.987111+0000","last_peered":"2023-10-16T08:16:26.987111+0000","last_clean":"2023-10-16T08:16:26.987111+0000","last_became_active":"2023-10-16T08:16:22.964839+0000","last_became_peered":"2023-10-16T08:16:22.964839+0000","last_unstale":"2023-10-16T08:16:26.987111+0000","last_undegraded":"2023-10-16T08:16:26.987111+0000","last_fullsized":"2023-10-16T08:16:26.987111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.978846+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.975261+0000","last_became_peered":"2023-10-16T08:16:22.975261+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996150+0000","last_change":"2023-10-16T08:16:22.973907+0000","last_active":"2023-10-16T08:16:26.996150+0000","last_peered":"2023-10-16T08:16:26.996150+0000","last_clean":"2023-10-16T08:16:26.996150+0000","last_became_active":"2023-10-16T08:16:22.973609+0000","last_became_peered":"2023-10-16T08:16:22.973609+0000","last_unstale":"2023-10-16T08:16:26.996150+0000","last_undegraded":"2023-10-16T08:16:26.996150+0000","last_fullsized":"2023-10-16T08:16:26.996150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994986+0000","last_change":"2023-10-16T08:16:22.995822+0000","last_active":"2023-10-16T08:16:26.994986+0000","last_peered":"2023-10-16T08:16:26.994986+0000","last_clean":"2023-10-16T08:16:26.994986+0000","last_became_active":"2023-10-16T08:16:22.995473+0000","last_became_peered":"2023-10-16T08:16:22.995473+0000","last_unstale":"2023-10-16T08:16:26.994986+0000","last_undegraded":"2023-10-16T08:16:26.994986+0000","last_fullsized":"2023-10-16T08:16:26.994986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994188+0000","last_change":"2023-10-16T08:16:22.975976+0000","last_active":"2023-10-16T08:16:26.994188+0000","last_peered":"2023-10-16T08:16:26.994188+0000","last_clean":"2023-10-16T08:16:26.994188+0000","last_became_active":"2023-10-16T08:16:22.975600+0000","last_became_peered":"2023-10-16T08:16:22.975600+0000","last_unstale":"2023-10-16T08:16:26.994188+0000","last_undegraded":"2023-10-16T08:16:26.994188+0000","last_fullsized":"2023-10-16T08:16:26.994188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009586+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.009586+0000","last_peered":"2023-10-16T08:16:27.009586+0000","last_clean":"2023-10-16T08:16:27.009586+0000","last_became_active":"2023-10-16T08:16:22.954170+0000","last_became_peered":"2023-10-16T08:16:22.954170+0000","last_unstale":"2023-10-16T08:16:27.009586+0000","last_undegraded":"2023-10-16T08:16:27.009586+0000","last_fullsized":"2023-10-16T08:16:27.009586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984253+0000","last_change":"2023-10-16T08:16:22.959029+0000","last_active":"2023-10-16T08:16:26.984253+0000","last_peered":"2023-10-16T08:16:26.984253+0000","last_clean":"2023-10-16T08:16:26.984253+0000","last_became_active":"2023-10-16T08:16:22.958713+0000","last_became_peered":"2023-10-16T08:16:22.958713+0000","last_unstale":"2023-10-16T08:16:26.984253+0000","last_undegraded":"2023-10-16T08:16:26.984253+0000","last_fullsized":"2023-10-16T08:16:26.984253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:47.145986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990281+0000","last_change":"2023-10-16T08:16:22.973255+0000","last_active":"2023-10-16T08:16:26.990281+0000","last_peered":"2023-10-16T08:16:26.990281+0000","last_clean":"2023-10-16T08:16:26.990281+0000","last_became_active":"2023-10-16T08:16:22.972582+0000","last_became_peered":"2023-10-16T08:16:22.972582+0000","last_unstale":"2023-10-16T08:16:26.990281+0000","last_undegraded":"2023-10-16T08:16:26.990281+0000","last_fullsized":"2023-10-16T08:16:26.990281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004704+0000","last_change":"2023-10-16T08:16:22.969190+0000","last_active":"2023-10-16T08:16:27.004704+0000","last_peered":"2023-10-16T08:16:27.004704+0000","last_clean":"2023-10-16T08:16:27.004704+0000","last_became_active":"2023-10-16T08:16:22.968880+0000","last_became_peered":"2023-10-16T08:16:22.968880+0000","last_unstale":"2023-10-16T08:16:27.004704+0000","last_undegraded":"2023-10-16T08:16:27.004704+0000","last_fullsized":"2023-10-16T08:16:27.004704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987745+0000","last_change":"2023-10-16T08:16:22.975463+0000","last_active":"2023-10-16T08:16:26.987745+0000","last_peered":"2023-10-16T08:16:26.987745+0000","last_clean":"2023-10-16T08:16:26.987745+0000","last_became_active":"2023-10-16T08:16:22.975080+0000","last_became_peered":"2023-10-16T08:16:22.975080+0000","last_unstale":"2023-10-16T08:16:26.987745+0000","last_undegraded":"2023-10-16T08:16:26.987745+0000","last_fullsized":"2023-10-16T08:16:26.987745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975707+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.975429+0000","last_became_peered":"2023-10-16T08:16:22.975429+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179066+0000","last_change":"2023-10-16T08:16:22.977921+0000","last_active":"2023-10-16T08:16:27.179066+0000","last_peered":"2023-10-16T08:16:27.179066+0000","last_clean":"2023-10-16T08:16:27.179066+0000","last_became_active":"2023-10-16T08:16:22.974938+0000","last_became_peered":"2023-10-16T08:16:22.974938+0000","last_unstale":"2023-10-16T08:16:27.179066+0000","last_undegraded":"2023-10-16T08:16:27.179066+0000","last_fullsized":"2023-10-16T08:16:27.179066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:35.197545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986251+0000","last_change":"2023-10-16T08:16:22.972026+0000","last_active":"2023-10-16T08:16:26.986251+0000","last_peered":"2023-10-16T08:16:26.986251+0000","last_clean":"2023-10-16T08:16:26.986251+0000","last_became_active":"2023-10-16T08:16:22.971655+0000","last_became_peered":"2023-10-16T08:16:22.971655+0000","last_unstale":"2023-10-16T08:16:26.986251+0000","last_undegraded":"2023-10-16T08:16:26.986251+0000","last_fullsized":"2023-10-16T08:16:26.986251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178078+0000","last_change":"2023-10-16T08:16:22.973377+0000","last_active":"2023-10-16T08:16:27.178078+0000","last_peered":"2023-10-16T08:16:27.178078+0000","last_clean":"2023-10-16T08:16:27.178078+0000","last_became_active":"2023-10-16T08:16:22.972888+0000","last_became_peered":"2023-10-16T08:16:22.972888+0000","last_unstale":"2023-10-16T08:16:27.178078+0000","last_undegraded":"2023-10-16T08:16:27.178078+0000","last_fullsized":"2023-10-16T08:16:27.178078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994140+0000","last_change":"2023-10-16T08:16:22.976999+0000","last_active":"2023-10-16T08:16:26.994140+0000","last_peered":"2023-10-16T08:16:26.994140+0000","last_clean":"2023-10-16T08:16:26.994140+0000","last_became_active":"2023-10-16T08:16:22.974943+0000","last_became_peered":"2023-10-16T08:16:22.974943+0000","last_unstale":"2023-10-16T08:16:26.994140+0000","last_undegraded":"2023-10-16T08:16:26.994140+0000","last_fullsized":"2023-10-16T08:16:26.994140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178298+0000","last_change":"2023-10-16T08:16:22.979562+0000","last_active":"2023-10-16T08:16:27.178298+0000","last_peered":"2023-10-16T08:16:27.178298+0000","last_clean":"2023-10-16T08:16:27.178298+0000","last_became_active":"2023-10-16T08:16:22.975468+0000","last_became_peered":"2023-10-16T08:16:22.975468+0000","last_unstale":"2023-10-16T08:16:27.178298+0000","last_undegraded":"2023-10-16T08:16:27.178298+0000","last_fullsized":"2023-10-16T08:16:27.178298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994788+0000","last_change":"2023-10-16T08:16:22.991415+0000","last_active":"2023-10-16T08:16:26.994788+0000","last_peered":"2023-10-16T08:16:26.994788+0000","last_clean":"2023-10-16T08:16:26.994788+0000","last_became_active":"2023-10-16T08:16:22.990410+0000","last_became_peered":"2023-10-16T08:16:22.990410+0000","last_unstale":"2023-10-16T08:16:26.994788+0000","last_undegraded":"2023-10-16T08:16:26.994788+0000","last_fullsized":"2023-10-16T08:16:26.994788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987345+0000","last_change":"2023-10-16T08:16:22.957927+0000","last_active":"2023-10-16T08:16:26.987345+0000","last_peered":"2023-10-16T08:16:26.987345+0000","last_clean":"2023-10-16T08:16:26.987345+0000","last_became_active":"2023-10-16T08:16:22.956909+0000","last_became_peered":"2023-10-16T08:16:22.956909+0000","last_unstale":"2023-10-16T08:16:26.987345+0000","last_undegraded":"2023-10-16T08:16:26.987345+0000","last_fullsized":"2023-10-16T08:16:26.987345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014243+0000","last_change":"2023-10-16T08:16:22.963419+0000","last_active":"2023-10-16T08:16:27.014243+0000","last_peered":"2023-10-16T08:16:27.014243+0000","last_clean":"2023-10-16T08:16:27.014243+0000","last_became_active":"2023-10-16T08:16:22.962692+0000","last_became_peered":"2023-10-16T08:16:22.962692+0000","last_unstale":"2023-10-16T08:16:27.014243+0000","last_undegraded":"2023-10-16T08:16:27.014243+0000","last_fullsized":"2023-10-16T08:16:27.014243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178991+0000","last_change":"2023-10-16T08:16:22.976621+0000","last_active":"2023-10-16T08:16:27.178991+0000","last_peered":"2023-10-16T08:16:27.178991+0000","last_clean":"2023-10-16T08:16:27.178991+0000","last_became_active":"2023-10-16T08:16:22.975391+0000","last_became_peered":"2023-10-16T08:16:22.975391+0000","last_unstale":"2023-10-16T08:16:27.178991+0000","last_undegraded":"2023-10-16T08:16:27.178991+0000","last_fullsized":"2023-10-16T08:16:27.178991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:16:47.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:16:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 Processor -- start 2023-10-16T08:16:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 -- start start 2023-10-16T08:16:48.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a2415cb90 0x7f0a240cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0a240d15d0 0x7f0a240c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a240ce770 con 0x7f0a240d3010 2023-10-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a240d0600 con 0x7f0a2415cb90 2023-10-16T08:16:48.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a2952a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a240c6450 con 0x7f0a240d15d0 2023-10-16T08:16:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.102+0000 7f0a22ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a2415cb90 0x7f0a240cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a1bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a240d3010 0x7f0a240d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44398/0 (socket says 172.21.15.32:44398) 2023-10-16T08:16:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a1bfff640 1 -- 172.21.15.32:0/1646353765 learned_addr learned my addr 172.21.15.32:0/1646353765 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a237fe640 1 --2- 172.21.15.32:0/1646353765 >> v2:172.21.15.186:3300/0 conn(0x7f0a240d15d0 0x7f0a240c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a22ffd640 1 -- 172.21.15.32:0/1646353765 >> v2:172.21.15.186:3300/0 conn(0x7f0a240d15d0 msgr2=0x7f0a240c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a22ffd640 1 --2- 172.21.15.32:0/1646353765 >> v2:172.21.15.186:3300/0 conn(0x7f0a240d15d0 0x7f0a240c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a22ffd640 1 -- 172.21.15.32:0/1646353765 >> v1:172.21.15.32:6789/0 conn(0x7f0a240d3010 legacy=0x7f0a240d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a22ffd640 1 -- 172.21.15.32:0/1646353765 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240d08a0 con 0x7f0a2415cb90 2023-10-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a227fc640 1 -- 172.21.15.32:0/1646353765 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627856714 0 0) 0x7f0a240ce770 con 0x7f0a240d3010 2023-10-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a22ffd640 1 --2- 172.21.15.32:0/1646353765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a240cfb10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a240ccee0 tx=0x7f0a1401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c51f8dca20c6c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a227fc640 1 -- 172.21.15.32:0/1646353765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140ad680 con 0x7f0a2415cb90 2023-10-16T08:16:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a227fc640 1 -- 172.21.15.32:0/1646353765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a140ad7e0 con 0x7f0a2415cb90 2023-10-16T08:16:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.106+0000 7f0a227fc640 1 -- 172.21.15.32:0/1646353765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140adb70 con 0x7f0a2415cb90 2023-10-16T08:16:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- 172.21.15.32:0/1646353765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 msgr2=0x7f0a240cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1646353765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a240cfb10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a240ccee0 tx=0x7f0a1401c380 comp rx=0 tx=0).stop 2023-10-16T08:16:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- 172.21.15.32:0/1646353765 shutdown_connections 2023-10-16T08:16:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1646353765 >> v2:172.21.15.186:3300/0 conn(0x7f0a240d15d0 0x7f0a240c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1646353765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a240cfb10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- 172.21.15.32:0/1646353765 >> 172.21.15.32:0/1646353765 conn(0x7f0a2409bc50 msgr2=0x7f0a240d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- 172.21.15.32:0/1646353765 shutdown_connections 2023-10-16T08:16:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- 172.21.15.32:0/1646353765 wait complete. 2023-10-16T08:16:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 Processor -- start 2023-10-16T08:16:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- start start 2023-10-16T08:16:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a240d15d0 0x7f0a24157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a240d3010 0x7f0a240b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a24151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a241599f0 con 0x7f0a2415cb90 2023-10-16T08:16:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24157d40 con 0x7f0a240d3010 2023-10-16T08:16:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a2952a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240b5db0 con 0x7f0a240d15d0 2023-10-16T08:16:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a24151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a24151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58970/0 (socket says 172.21.15.32:58970) 2023-10-16T08:16:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 -- 172.21.15.32:0/1681512033 learned_addr learned my addr 172.21.15.32:0/1681512033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a1bfff640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a240d3010 0x7f0a240b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a22ffd640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a240d15d0 0x7f0a24157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 -- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a240d15d0 msgr2=0x7f0a24157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a240d15d0 0x7f0a24157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 -- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a240d3010 msgr2=0x7f0a240b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a240d3010 0x7f0a240b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a237fe640 1 -- 172.21.15.32:0/1681512033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140093b0 con 0x7f0a2415cb90 2023-10-16T08:16:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.110+0000 7f0a1bfff640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a240d3010 0x7f0a240b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a237fe640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a24151870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009dd0 tx=0x7f0a1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c014480 con 0x7f0a2415cb90 2023-10-16T08:16:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c00f040 con 0x7f0a2415cb90 2023-10-16T08:16:48.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c014840 con 0x7f0a2415cb90 2023-10-16T08:16:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a24159c40 con 0x7f0a2415cb90 2023-10-16T08:16:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2415a200 con 0x7f0a2415cb90 2023-10-16T08:16:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a24074010 con 0x7f0a2415cb90 2023-10-16T08:16:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a1c0149a0 con 0x7f0a2415cb90 2023-10-16T08:16:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a1b7fe640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0a040664a0 0x7f0a04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a1c084da0 con 0x7f0a2415cb90 2023-10-16T08:16:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.114+0000 7f0a22ffd640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0a040664a0 0x7f0a04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.118+0000 7f0a22ffd640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0a040664a0 0x7f0a04068960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a140af000 tx=0x7f0a140a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.118+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a1c051e90 con 0x7f0a2415cb90 2023-10-16T08:16:48.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.274+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a2415cf70 con 0x7f0a040664a0 2023-10-16T08:16:48.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.282+0000 7f0a1b7fe640 1 -- 172.21.15.32:0/1681512033 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345290 (secure 0 0 0) 0x7f0a140ad950 con 0x7f0a040664a0 2023-10-16T08:16:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:16:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0a040664a0 msgr2=0x7f0a04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0a040664a0 0x7f0a04068960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a140af000 tx=0x7f0a140a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 msgr2=0x7f0a24151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a24151870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009dd0 tx=0x7f0a1c00c490 comp rx=0 tx=0).stop 2023-10-16T08:16:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 shutdown_connections 2023-10-16T08:16:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2415cb90 0x7f0a24151870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a240d3010 0x7f0a240b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0a040664a0 0x7f0a04068960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 --2- 172.21.15.32:0/1681512033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0a240d15d0 0x7f0a24157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 >> 172.21.15.32:0/1681512033 conn(0x7f0a2409bc50 msgr2=0x7f0a240cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 shutdown_connections 2023-10-16T08:16:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.286+0000 7f0a2952a640 1 -- 172.21.15.32:0/1681512033 wait complete. 2023-10-16T08:16:48.313 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-10-16T08:16:48.019608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223628,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751092,"statfs":{"total":773094113280,"available":772865118208,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":225496635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003316"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988248+0000","last_change":"2023-10-16T08:16:21.956684+0000","last_active":"2023-10-16T08:16:26.988248+0000","last_peered":"2023-10-16T08:16:26.988248+0000","last_clean":"2023-10-16T08:16:26.988248+0000","last_became_active":"2023-10-16T08:16:21.949061+0000","last_became_peered":"2023-10-16T08:16:21.949061+0000","last_unstale":"2023-10-16T08:16:26.988248+0000","last_undegraded":"2023-10-16T08:16:26.988248+0000","last_fullsized":"2023-10-16T08:16:26.988248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985413+0000","last_change":"2023-10-16T08:16:22.972913+0000","last_active":"2023-10-16T08:16:26.985413+0000","last_peered":"2023-10-16T08:16:26.985413+0000","last_clean":"2023-10-16T08:16:26.985413+0000","last_became_active":"2023-10-16T08:16:22.972605+0000","last_became_peered":"2023-10-16T08:16:22.972605+0000","last_unstale":"2023-10-16T08:16:26.985413+0000","last_undegraded":"2023-10-16T08:16:26.985413+0000","last_fullsized":"2023-10-16T08:16:26.985413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989573+0000","last_change":"2023-10-16T08:16:27.182406+0000","last_active":"2023-10-16T08:16:27.989573+0000","last_peered":"2023-10-16T08:16:27.989573+0000","last_clean":"2023-10-16T08:16:27.989573+0000","last_became_active":"2023-10-16T08:16:27.182005+0000","last_became_peered":"2023-10-16T08:16:27.182005+0000","last_unstale":"2023-10-16T08:16:27.989573+0000","last_undegraded":"2023-10-16T08:16:27.989573+0000","last_fullsized":"2023-10-16T08:16:27.989573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:06.143173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985684+0000","last_change":"2023-10-16T08:16:21.945969+0000","last_active":"2023-10-16T08:16:26.985684+0000","last_peered":"2023-10-16T08:16:26.985684+0000","last_clean":"2023-10-16T08:16:26.985684+0000","last_became_active":"2023-10-16T08:16:21.945681+0000","last_became_peered":"2023-10-16T08:16:21.945681+0000","last_unstale":"2023-10-16T08:16:26.985684+0000","last_undegraded":"2023-10-16T08:16:26.985684+0000","last_fullsized":"2023-10-16T08:16:26.985684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988041+0000","last_change":"2023-10-16T08:16:22.974743+0000","last_active":"2023-10-16T08:16:26.988041+0000","last_peered":"2023-10-16T08:16:26.988041+0000","last_clean":"2023-10-16T08:16:26.988041+0000","last_became_active":"2023-10-16T08:16:22.974180+0000","last_became_peered":"2023-10-16T08:16:22.974180+0000","last_unstale":"2023-10-16T08:16:26.988041+0000","last_undegraded":"2023-10-16T08:16:26.988041+0000","last_fullsized":"2023-10-16T08:16:26.988041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542410+0000","last_change":"2023-10-16T08:16:27.011704+0000","last_active":"2023-10-16T08:16:29.542410+0000","last_peered":"2023-10-16T08:16:29.542410+0000","last_clean":"2023-10-16T08:16:29.542410+0000","last_became_active":"2023-10-16T08:16:27.011142+0000","last_became_peered":"2023-10-16T08:16:27.011142+0000","last_unstale":"2023-10-16T08:16:29.542410+0000","last_undegraded":"2023-10-16T08:16:29.542410+0000","last_fullsized":"2023-10-16T08:16:29.542410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:17.770482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178836+0000","last_change":"2023-10-16T08:16:21.964716+0000","last_active":"2023-10-16T08:16:27.178836+0000","last_peered":"2023-10-16T08:16:27.178836+0000","last_clean":"2023-10-16T08:16:27.178836+0000","last_became_active":"2023-10-16T08:16:21.964451+0000","last_became_peered":"2023-10-16T08:16:21.964451+0000","last_unstale":"2023-10-16T08:16:27.178836+0000","last_undegraded":"2023-10-16T08:16:27.178836+0000","last_fullsized":"2023-10-16T08:16:27.178836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994868+0000","last_change":"2023-10-16T08:16:22.971839+0000","last_active":"2023-10-16T08:16:26.994868+0000","last_peered":"2023-10-16T08:16:26.994868+0000","last_clean":"2023-10-16T08:16:26.994868+0000","last_became_active":"2023-10-16T08:16:22.971456+0000","last_became_peered":"2023-10-16T08:16:22.971456+0000","last_unstale":"2023-10-16T08:16:26.994868+0000","last_undegraded":"2023-10-16T08:16:26.994868+0000","last_fullsized":"2023-10-16T08:16:26.994868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543415+0000","last_change":"2023-10-16T08:16:27.024297+0000","last_active":"2023-10-16T08:16:29.543415+0000","last_peered":"2023-10-16T08:16:29.543415+0000","last_clean":"2023-10-16T08:16:29.543415+0000","last_became_active":"2023-10-16T08:16:27.023792+0000","last_became_peered":"2023-10-16T08:16:27.023792+0000","last_unstale":"2023-10-16T08:16:29.543415+0000","last_undegraded":"2023-10-16T08:16:29.543415+0000","last_fullsized":"2023-10-16T08:16:29.543415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:20.519605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178617+0000","last_change":"2023-10-16T08:16:21.964025+0000","last_active":"2023-10-16T08:16:27.178617+0000","last_peered":"2023-10-16T08:16:27.178617+0000","last_clean":"2023-10-16T08:16:27.178617+0000","last_became_active":"2023-10-16T08:16:21.963582+0000","last_became_peered":"2023-10-16T08:16:21.963582+0000","last_unstale":"2023-10-16T08:16:27.178617+0000","last_undegraded":"2023-10-16T08:16:27.178617+0000","last_fullsized":"2023-10-16T08:16:27.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994775+0000","last_change":"2023-10-16T08:16:22.980735+0000","last_active":"2023-10-16T08:16:26.994775+0000","last_peered":"2023-10-16T08:16:26.994775+0000","last_clean":"2023-10-16T08:16:26.994775+0000","last_became_active":"2023-10-16T08:16:22.980409+0000","last_became_peered":"2023-10-16T08:16:22.980409+0000","last_unstale":"2023-10-16T08:16:26.994775+0000","last_undegraded":"2023-10-16T08:16:26.994775+0000","last_fullsized":"2023-10-16T08:16:26.994775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542893+0000","last_change":"2023-10-16T08:16:27.024269+0000","last_active":"2023-10-16T08:16:29.542893+0000","last_peered":"2023-10-16T08:16:29.542893+0000","last_clean":"2023-10-16T08:16:29.542893+0000","last_became_active":"2023-10-16T08:16:27.023822+0000","last_became_peered":"2023-10-16T08:16:27.023822+0000","last_unstale":"2023-10-16T08:16:29.542893+0000","last_undegraded":"2023-10-16T08:16:29.542893+0000","last_fullsized":"2023-10-16T08:16:29.542893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:49.956062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987114+0000","last_change":"2023-10-16T08:16:22.969401+0000","last_active":"2023-10-16T08:16:26.987114+0000","last_peered":"2023-10-16T08:16:26.987114+0000","last_clean":"2023-10-16T08:16:26.987114+0000","last_became_active":"2023-10-16T08:16:22.969099+0000","last_became_peered":"2023-10-16T08:16:22.969099+0000","last_unstale":"2023-10-16T08:16:26.987114+0000","last_undegraded":"2023-10-16T08:16:26.987114+0000","last_fullsized":"2023-10-16T08:16:26.987114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178720+0000","last_change":"2023-10-16T08:16:21.961388+0000","last_active":"2023-10-16T08:16:27.178720+0000","last_peered":"2023-10-16T08:16:27.178720+0000","last_clean":"2023-10-16T08:16:27.178720+0000","last_became_active":"2023-10-16T08:16:21.961050+0000","last_became_peered":"2023-10-16T08:16:21.961050+0000","last_unstale":"2023-10-16T08:16:27.178720+0000","last_undegraded":"2023-10-16T08:16:27.178720+0000","last_fullsized":"2023-10-16T08:16:27.178720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:13.387501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543302+0000","last_change":"2023-10-16T08:16:27.019072+0000","last_active":"2023-10-16T08:16:29.543302+0000","last_peered":"2023-10-16T08:16:29.543302+0000","last_clean":"2023-10-16T08:16:29.543302+0000","last_became_active":"2023-10-16T08:16:27.018440+0000","last_became_peered":"2023-10-16T08:16:27.018440+0000","last_unstale":"2023-10-16T08:16:29.543302+0000","last_undegraded":"2023-10-16T08:16:29.543302+0000","last_fullsized":"2023-10-16T08:16:29.543302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:39.310251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990373+0000","last_change":"2023-10-16T08:16:22.960667+0000","last_active":"2023-10-16T08:16:26.990373+0000","last_peered":"2023-10-16T08:16:26.990373+0000","last_clean":"2023-10-16T08:16:26.990373+0000","last_became_active":"2023-10-16T08:16:22.960345+0000","last_became_peered":"2023-10-16T08:16:22.960345+0000","last_unstale":"2023-10-16T08:16:26.990373+0000","last_undegraded":"2023-10-16T08:16:26.990373+0000","last_fullsized":"2023-10-16T08:16:26.990373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013495+0000","last_change":"2023-10-16T08:16:21.950408+0000","last_active":"2023-10-16T08:16:27.013495+0000","last_peered":"2023-10-16T08:16:27.013495+0000","last_clean":"2023-10-16T08:16:27.013495+0000","last_became_active":"2023-10-16T08:16:21.947942+0000","last_became_peered":"2023-10-16T08:16:21.947942+0000","last_unstale":"2023-10-16T08:16:27.013495+0000","last_undegraded":"2023-10-16T08:16:27.013495+0000","last_fullsized":"2023-10-16T08:16:27.013495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542836+0000","last_change":"2023-10-16T08:16:27.021686+0000","last_active":"2023-10-16T08:16:29.542836+0000","last_peered":"2023-10-16T08:16:29.542836+0000","last_clean":"2023-10-16T08:16:29.542836+0000","last_became_active":"2023-10-16T08:16:27.021230+0000","last_became_peered":"2023-10-16T08:16:27.021230+0000","last_unstale":"2023-10-16T08:16:29.542836+0000","last_undegraded":"2023-10-16T08:16:29.542836+0000","last_fullsized":"2023-10-16T08:16:29.542836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:12.583792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986505+0000","last_change":"2023-10-16T08:16:22.955965+0000","last_active":"2023-10-16T08:16:26.986505+0000","last_peered":"2023-10-16T08:16:26.986505+0000","last_clean":"2023-10-16T08:16:26.986505+0000","last_became_active":"2023-10-16T08:16:22.955652+0000","last_became_peered":"2023-10-16T08:16:22.955652+0000","last_unstale":"2023-10-16T08:16:26.986505+0000","last_undegraded":"2023-10-16T08:16:26.986505+0000","last_fullsized":"2023-10-16T08:16:26.986505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009595+0000","last_change":"2023-10-16T08:16:21.950219+0000","last_active":"2023-10-16T08:16:27.009595+0000","last_peered":"2023-10-16T08:16:27.009595+0000","last_clean":"2023-10-16T08:16:27.009595+0000","last_became_active":"2023-10-16T08:16:21.949749+0000","last_became_peered":"2023-10-16T08:16:21.949749+0000","last_unstale":"2023-10-16T08:16:27.009595+0000","last_undegraded":"2023-10-16T08:16:27.009595+0000","last_fullsized":"2023-10-16T08:16:27.009595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988141+0000","last_change":"2023-10-16T08:16:27.182341+0000","last_active":"2023-10-16T08:16:27.988141+0000","last_peered":"2023-10-16T08:16:27.988141+0000","last_clean":"2023-10-16T08:16:27.988141+0000","last_became_active":"2023-10-16T08:16:27.181854+0000","last_became_peered":"2023-10-16T08:16:27.181854+0000","last_unstale":"2023-10-16T08:16:27.988141+0000","last_undegraded":"2023-10-16T08:16:27.988141+0000","last_fullsized":"2023-10-16T08:16:27.988141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003825+0000","last_change":"2023-10-16T08:16:22.962471+0000","last_active":"2023-10-16T08:16:27.003825+0000","last_peered":"2023-10-16T08:16:27.003825+0000","last_clean":"2023-10-16T08:16:27.003825+0000","last_became_active":"2023-10-16T08:16:22.961915+0000","last_became_peered":"2023-10-16T08:16:22.961915+0000","last_unstale":"2023-10-16T08:16:27.003825+0000","last_undegraded":"2023-10-16T08:16:27.003825+0000","last_fullsized":"2023-10-16T08:16:27.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:13.023207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177873+0000","last_change":"2023-10-16T08:16:21.964429+0000","last_active":"2023-10-16T08:16:27.177873+0000","last_peered":"2023-10-16T08:16:27.177873+0000","last_clean":"2023-10-16T08:16:27.177873+0000","last_became_active":"2023-10-16T08:16:21.964154+0000","last_became_peered":"2023-10-16T08:16:21.964154+0000","last_unstale":"2023-10-16T08:16:27.177873+0000","last_undegraded":"2023-10-16T08:16:27.177873+0000","last_fullsized":"2023-10-16T08:16:27.177873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543629+0000","last_change":"2023-10-16T08:16:27.027359+0000","last_active":"2023-10-16T08:16:29.543629+0000","last_peered":"2023-10-16T08:16:29.543629+0000","last_clean":"2023-10-16T08:16:29.543629+0000","last_became_active":"2023-10-16T08:16:27.026923+0000","last_became_peered":"2023-10-16T08:16:27.026923+0000","last_unstale":"2023-10-16T08:16:29.543629+0000","last_undegraded":"2023-10-16T08:16:29.543629+0000","last_fullsized":"2023-10-16T08:16:29.543629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.989245+0000","last_change":"2023-10-16T08:16:22.967177+0000","last_active":"2023-10-16T08:16:26.989245+0000","last_peered":"2023-10-16T08:16:26.989245+0000","last_clean":"2023-10-16T08:16:26.989245+0000","last_became_active":"2023-10-16T08:16:22.966812+0000","last_became_peered":"2023-10-16T08:16:22.966812+0000","last_unstale":"2023-10-16T08:16:26.989245+0000","last_undegraded":"2023-10-16T08:16:26.989245+0000","last_fullsized":"2023-10-16T08:16:26.989245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988555+0000","last_change":"2023-10-16T08:16:21.958338+0000","last_active":"2023-10-16T08:16:26.988555+0000","last_peered":"2023-10-16T08:16:26.988555+0000","last_clean":"2023-10-16T08:16:26.988555+0000","last_became_active":"2023-10-16T08:16:21.958052+0000","last_became_peered":"2023-10-16T08:16:21.958052+0000","last_unstale":"2023-10-16T08:16:26.988555+0000","last_undegraded":"2023-10-16T08:16:26.988555+0000","last_fullsized":"2023-10-16T08:16:26.988555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543530+0000","last_change":"2023-10-16T08:16:27.184138+0000","last_active":"2023-10-16T08:16:29.543530+0000","last_peered":"2023-10-16T08:16:29.543530+0000","last_clean":"2023-10-16T08:16:29.543530+0000","last_became_active":"2023-10-16T08:16:27.183769+0000","last_became_peered":"2023-10-16T08:16:27.183769+0000","last_unstale":"2023-10-16T08:16:29.543530+0000","last_undegraded":"2023-10-16T08:16:29.543530+0000","last_fullsized":"2023-10-16T08:16:29.543530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987117+0000","last_change":"2023-10-16T08:16:22.973708+0000","last_active":"2023-10-16T08:16:26.987117+0000","last_peered":"2023-10-16T08:16:26.987117+0000","last_clean":"2023-10-16T08:16:26.987117+0000","last_became_active":"2023-10-16T08:16:22.973402+0000","last_became_peered":"2023-10-16T08:16:22.973402+0000","last_unstale":"2023-10-16T08:16:26.987117+0000","last_undegraded":"2023-10-16T08:16:26.987117+0000","last_fullsized":"2023-10-16T08:16:26.987117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014342+0000","last_change":"2023-10-16T08:16:21.955020+0000","last_active":"2023-10-16T08:16:27.014342+0000","last_peered":"2023-10-16T08:16:27.014342+0000","last_clean":"2023-10-16T08:16:27.014342+0000","last_became_active":"2023-10-16T08:16:21.954681+0000","last_became_peered":"2023-10-16T08:16:21.954681+0000","last_unstale":"2023-10-16T08:16:27.014342+0000","last_undegraded":"2023-10-16T08:16:27.014342+0000","last_fullsized":"2023-10-16T08:16:27.014342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989486+0000","last_change":"2023-10-16T08:16:27.179578+0000","last_active":"2023-10-16T08:16:27.989486+0000","last_peered":"2023-10-16T08:16:27.989486+0000","last_clean":"2023-10-16T08:16:27.989486+0000","last_became_active":"2023-10-16T08:16:27.178683+0000","last_became_peered":"2023-10-16T08:16:27.178683+0000","last_unstale":"2023-10-16T08:16:27.989486+0000","last_undegraded":"2023-10-16T08:16:27.989486+0000","last_fullsized":"2023-10-16T08:16:27.989486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014216+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.014216+0000","last_peered":"2023-10-16T08:16:27.014216+0000","last_clean":"2023-10-16T08:16:27.014216+0000","last_became_active":"2023-10-16T08:16:22.952824+0000","last_became_peered":"2023-10-16T08:16:22.952824+0000","last_unstale":"2023-10-16T08:16:27.014216+0000","last_undegraded":"2023-10-16T08:16:27.014216+0000","last_fullsized":"2023-10-16T08:16:27.014216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178889+0000","last_change":"2023-10-16T08:16:21.964665+0000","last_active":"2023-10-16T08:16:27.178889+0000","last_peered":"2023-10-16T08:16:27.178889+0000","last_clean":"2023-10-16T08:16:27.178889+0000","last_became_active":"2023-10-16T08:16:21.964105+0000","last_became_peered":"2023-10-16T08:16:21.964105+0000","last_unstale":"2023-10-16T08:16:27.178889+0000","last_undegraded":"2023-10-16T08:16:27.178889+0000","last_fullsized":"2023-10-16T08:16:27.178889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:12.709836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541103+0000","last_change":"2023-10-16T08:16:27.180634+0000","last_active":"2023-10-16T08:16:29.541103+0000","last_peered":"2023-10-16T08:16:29.541103+0000","last_clean":"2023-10-16T08:16:29.541103+0000","last_became_active":"2023-10-16T08:16:27.180186+0000","last_became_peered":"2023-10-16T08:16:27.180186+0000","last_unstale":"2023-10-16T08:16:29.541103+0000","last_undegraded":"2023-10-16T08:16:29.541103+0000","last_fullsized":"2023-10-16T08:16:29.541103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994880+0000","last_change":"2023-10-16T08:16:22.994702+0000","last_active":"2023-10-16T08:16:26.994880+0000","last_peered":"2023-10-16T08:16:26.994880+0000","last_clean":"2023-10-16T08:16:26.994880+0000","last_became_active":"2023-10-16T08:16:22.990441+0000","last_became_peered":"2023-10-16T08:16:22.990441+0000","last_unstale":"2023-10-16T08:16:26.994880+0000","last_undegraded":"2023-10-16T08:16:26.994880+0000","last_fullsized":"2023-10-16T08:16:26.994880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014301+0000","last_change":"2023-10-16T08:16:21.942771+0000","last_active":"2023-10-16T08:16:27.014301+0000","last_peered":"2023-10-16T08:16:27.014301+0000","last_clean":"2023-10-16T08:16:27.014301+0000","last_became_active":"2023-10-16T08:16:21.942441+0000","last_became_peered":"2023-10-16T08:16:21.942441+0000","last_unstale":"2023-10-16T08:16:27.014301+0000","last_undegraded":"2023-10-16T08:16:27.014301+0000","last_fullsized":"2023-10-16T08:16:27.014301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.986815+0000","last_change":"2023-10-16T08:16:27.013697+0000","last_active":"2023-10-16T08:16:27.986815+0000","last_peered":"2023-10-16T08:16:27.986815+0000","last_clean":"2023-10-16T08:16:27.986815+0000","last_became_active":"2023-10-16T08:16:27.012937+0000","last_became_peered":"2023-10-16T08:16:27.012937+0000","last_unstale":"2023-10-16T08:16:27.986815+0000","last_undegraded":"2023-10-16T08:16:27.986815+0000","last_fullsized":"2023-10-16T08:16:27.986815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003417+0000","last_change":"2023-10-16T08:16:22.964069+0000","last_active":"2023-10-16T08:16:27.003417+0000","last_peered":"2023-10-16T08:16:27.003417+0000","last_clean":"2023-10-16T08:16:27.003417+0000","last_became_active":"2023-10-16T08:16:22.963802+0000","last_became_peered":"2023-10-16T08:16:22.963802+0000","last_unstale":"2023-10-16T08:16:27.003417+0000","last_undegraded":"2023-10-16T08:16:27.003417+0000","last_fullsized":"2023-10-16T08:16:27.003417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994257+0000","last_change":"2023-10-16T08:16:21.962674+0000","last_active":"2023-10-16T08:16:26.994257+0000","last_peered":"2023-10-16T08:16:26.994257+0000","last_clean":"2023-10-16T08:16:26.994257+0000","last_became_active":"2023-10-16T08:16:21.962316+0000","last_became_peered":"2023-10-16T08:16:21.962316+0000","last_unstale":"2023-10-16T08:16:26.994257+0000","last_undegraded":"2023-10-16T08:16:26.994257+0000","last_fullsized":"2023-10-16T08:16:26.994257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:32.032856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987533+0000","last_change":"2023-10-16T08:16:27.020747+0000","last_active":"2023-10-16T08:16:27.987533+0000","last_peered":"2023-10-16T08:16:27.987533+0000","last_clean":"2023-10-16T08:16:27.987533+0000","last_became_active":"2023-10-16T08:16:27.019634+0000","last_became_peered":"2023-10-16T08:16:27.019634+0000","last_unstale":"2023-10-16T08:16:27.987533+0000","last_undegraded":"2023-10-16T08:16:27.987533+0000","last_fullsized":"2023-10-16T08:16:27.987533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009729+0000","last_change":"2023-10-16T08:16:22.953894+0000","last_active":"2023-10-16T08:16:27.009729+0000","last_peered":"2023-10-16T08:16:27.009729+0000","last_clean":"2023-10-16T08:16:27.009729+0000","last_became_active":"2023-10-16T08:16:22.953606+0000","last_became_peered":"2023-10-16T08:16:22.953606+0000","last_unstale":"2023-10-16T08:16:27.009729+0000","last_undegraded":"2023-10-16T08:16:27.009729+0000","last_fullsized":"2023-10-16T08:16:27.009729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996654+0000","last_change":"2023-10-16T08:16:21.946996+0000","last_active":"2023-10-16T08:16:26.996654+0000","last_peered":"2023-10-16T08:16:26.996654+0000","last_clean":"2023-10-16T08:16:26.996654+0000","last_became_active":"2023-10-16T08:16:21.946675+0000","last_became_peered":"2023-10-16T08:16:21.946675+0000","last_unstale":"2023-10-16T08:16:26.996654+0000","last_undegraded":"2023-10-16T08:16:26.996654+0000","last_fullsized":"2023-10-16T08:16:26.996654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:43:55.303584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988192+0000","last_change":"2023-10-16T08:16:27.020112+0000","last_active":"2023-10-16T08:16:27.988192+0000","last_peered":"2023-10-16T08:16:27.988192+0000","last_clean":"2023-10-16T08:16:27.988192+0000","last_became_active":"2023-10-16T08:16:27.019606+0000","last_became_peered":"2023-10-16T08:16:27.019606+0000","last_unstale":"2023-10-16T08:16:27.988192+0000","last_undegraded":"2023-10-16T08:16:27.988192+0000","last_fullsized":"2023-10-16T08:16:27.988192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984195+0000","last_change":"2023-10-16T08:16:22.962655+0000","last_active":"2023-10-16T08:16:26.984195+0000","last_peered":"2023-10-16T08:16:26.984195+0000","last_clean":"2023-10-16T08:16:26.984195+0000","last_became_active":"2023-10-16T08:16:22.962344+0000","last_became_peered":"2023-10-16T08:16:22.962344+0000","last_unstale":"2023-10-16T08:16:26.984195+0000","last_undegraded":"2023-10-16T08:16:26.984195+0000","last_fullsized":"2023-10-16T08:16:26.984195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:43.247033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177937+0000","last_change":"2023-10-16T08:16:21.959357+0000","last_active":"2023-10-16T08:16:27.177937+0000","last_peered":"2023-10-16T08:16:27.177937+0000","last_clean":"2023-10-16T08:16:27.177937+0000","last_became_active":"2023-10-16T08:16:21.959015+0000","last_became_peered":"2023-10-16T08:16:21.959015+0000","last_unstale":"2023-10-16T08:16:27.177937+0000","last_undegraded":"2023-10-16T08:16:27.177937+0000","last_fullsized":"2023-10-16T08:16:27.177937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:02.223178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541384+0000","last_change":"2023-10-16T08:16:27.179195+0000","last_active":"2023-10-16T08:16:29.541384+0000","last_peered":"2023-10-16T08:16:29.541384+0000","last_clean":"2023-10-16T08:16:29.541384+0000","last_became_active":"2023-10-16T08:16:27.178732+0000","last_became_peered":"2023-10-16T08:16:27.178732+0000","last_unstale":"2023-10-16T08:16:29.541384+0000","last_undegraded":"2023-10-16T08:16:29.541384+0000","last_fullsized":"2023-10-16T08:16:29.541384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986752+0000","last_change":"2023-10-16T08:16:22.973605+0000","last_active":"2023-10-16T08:16:26.986752+0000","last_peered":"2023-10-16T08:16:26.986752+0000","last_clean":"2023-10-16T08:16:26.986752+0000","last_became_active":"2023-10-16T08:16:22.973281+0000","last_became_peered":"2023-10-16T08:16:22.973281+0000","last_unstale":"2023-10-16T08:16:26.986752+0000","last_undegraded":"2023-10-16T08:16:26.986752+0000","last_fullsized":"2023-10-16T08:16:26.986752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994407+0000","last_change":"2023-10-16T08:16:21.950850+0000","last_active":"2023-10-16T08:16:26.994407+0000","last_peered":"2023-10-16T08:16:26.994407+0000","last_clean":"2023-10-16T08:16:26.994407+0000","last_became_active":"2023-10-16T08:16:21.950526+0000","last_became_peered":"2023-10-16T08:16:21.950526+0000","last_unstale":"2023-10-16T08:16:26.994407+0000","last_undegraded":"2023-10-16T08:16:26.994407+0000","last_fullsized":"2023-10-16T08:16:26.994407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177966+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:16:27.177966+0000","last_peered":"2023-10-16T08:16:27.177966+0000","last_clean":"2023-10-16T08:16:27.177966+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:16:27.177966+0000","last_undegraded":"2023-10-16T08:16:27.177966+0000","last_fullsized":"2023-10-16T08:16:27.177966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991969+0000","last_change":"2023-10-16T08:16:27.184250+0000","last_active":"2023-10-16T08:16:27.991969+0000","last_peered":"2023-10-16T08:16:27.991969+0000","last_clean":"2023-10-16T08:16:27.991969+0000","last_became_active":"2023-10-16T08:16:27.183826+0000","last_became_peered":"2023-10-16T08:16:27.183826+0000","last_unstale":"2023-10-16T08:16:27.991969+0000","last_undegraded":"2023-10-16T08:16:27.991969+0000","last_fullsized":"2023-10-16T08:16:27.991969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178713+0000","last_change":"2023-10-16T08:16:22.975303+0000","last_active":"2023-10-16T08:16:27.178713+0000","last_peered":"2023-10-16T08:16:27.178713+0000","last_clean":"2023-10-16T08:16:27.178713+0000","last_became_active":"2023-10-16T08:16:22.974890+0000","last_became_peered":"2023-10-16T08:16:22.974890+0000","last_unstale":"2023-10-16T08:16:27.178713+0000","last_undegraded":"2023-10-16T08:16:27.178713+0000","last_fullsized":"2023-10-16T08:16:27.178713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.997283+0000","last_change":"2023-10-16T08:16:21.959337+0000","last_active":"2023-10-16T08:16:26.997283+0000","last_peered":"2023-10-16T08:16:26.997283+0000","last_clean":"2023-10-16T08:16:26.997283+0000","last_became_active":"2023-10-16T08:16:21.959059+0000","last_became_peered":"2023-10-16T08:16:21.959059+0000","last_unstale":"2023-10-16T08:16:26.997283+0000","last_undegraded":"2023-10-16T08:16:26.997283+0000","last_fullsized":"2023-10-16T08:16:26.997283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991525+0000","last_change":"2023-10-16T08:16:27.181051+0000","last_active":"2023-10-16T08:16:27.991525+0000","last_peered":"2023-10-16T08:16:27.991525+0000","last_clean":"2023-10-16T08:16:27.991525+0000","last_became_active":"2023-10-16T08:16:27.180503+0000","last_became_peered":"2023-10-16T08:16:27.180503+0000","last_unstale":"2023-10-16T08:16:27.991525+0000","last_undegraded":"2023-10-16T08:16:27.991525+0000","last_fullsized":"2023-10-16T08:16:27.991525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994965+0000","last_change":"2023-10-16T08:16:22.990070+0000","last_active":"2023-10-16T08:16:26.994965+0000","last_peered":"2023-10-16T08:16:26.994965+0000","last_clean":"2023-10-16T08:16:26.994965+0000","last_became_active":"2023-10-16T08:16:22.989787+0000","last_became_peered":"2023-10-16T08:16:22.989787+0000","last_unstale":"2023-10-16T08:16:26.994965+0000","last_undegraded":"2023-10-16T08:16:26.994965+0000","last_fullsized":"2023-10-16T08:16:26.994965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013982+0000","last_change":"2023-10-16T08:16:21.961287+0000","last_active":"2023-10-16T08:16:27.013982+0000","last_peered":"2023-10-16T08:16:27.013982+0000","last_clean":"2023-10-16T08:16:27.013982+0000","last_became_active":"2023-10-16T08:16:21.956293+0000","last_became_peered":"2023-10-16T08:16:21.956293+0000","last_unstale":"2023-10-16T08:16:27.013982+0000","last_undegraded":"2023-10-16T08:16:27.013982+0000","last_fullsized":"2023-10-16T08:16:27.013982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541257+0000","last_change":"2023-10-16T08:16:27.013097+0000","last_active":"2023-10-16T08:16:29.541257+0000","last_peered":"2023-10-16T08:16:29.541257+0000","last_clean":"2023-10-16T08:16:29.541257+0000","last_became_active":"2023-10-16T08:16:27.012587+0000","last_became_peered":"2023-10-16T08:16:27.012587+0000","last_unstale":"2023-10-16T08:16:29.541257+0000","last_undegraded":"2023-10-16T08:16:29.541257+0000","last_fullsized":"2023-10-16T08:16:29.541257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179067+0000","last_change":"2023-10-16T08:16:22.979584+0000","last_active":"2023-10-16T08:16:27.179067+0000","last_peered":"2023-10-16T08:16:27.179067+0000","last_clean":"2023-10-16T08:16:27.179067+0000","last_became_active":"2023-10-16T08:16:22.978239+0000","last_became_peered":"2023-10-16T08:16:22.978239+0000","last_unstale":"2023-10-16T08:16:27.179067+0000","last_undegraded":"2023-10-16T08:16:27.179067+0000","last_fullsized":"2023-10-16T08:16:27.179067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:27:57.200006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996247+0000","last_change":"2023-10-16T08:16:21.947037+0000","last_active":"2023-10-16T08:16:26.996247+0000","last_peered":"2023-10-16T08:16:26.996247+0000","last_clean":"2023-10-16T08:16:26.996247+0000","last_became_active":"2023-10-16T08:16:21.946671+0000","last_became_peered":"2023-10-16T08:16:21.946671+0000","last_unstale":"2023-10-16T08:16:26.996247+0000","last_undegraded":"2023-10-16T08:16:26.996247+0000","last_fullsized":"2023-10-16T08:16:26.996247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542197+0000","last_change":"2023-10-16T08:16:27.189477+0000","last_active":"2023-10-16T08:16:29.542197+0000","last_peered":"2023-10-16T08:16:29.542197+0000","last_clean":"2023-10-16T08:16:29.542197+0000","last_became_active":"2023-10-16T08:16:27.185942+0000","last_became_peered":"2023-10-16T08:16:27.185942+0000","last_unstale":"2023-10-16T08:16:29.542197+0000","last_undegraded":"2023-10-16T08:16:29.542197+0000","last_fullsized":"2023-10-16T08:16:29.542197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:13.645047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.973600+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.973215+0000","last_became_peered":"2023-10-16T08:16:22.973215+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.005005+0000","last_change":"2023-10-16T08:16:21.949416+0000","last_active":"2023-10-16T08:16:27.005005+0000","last_peered":"2023-10-16T08:16:27.005005+0000","last_clean":"2023-10-16T08:16:27.005005+0000","last_became_active":"2023-10-16T08:16:21.948836+0000","last_became_peered":"2023-10-16T08:16:21.948836+0000","last_unstale":"2023-10-16T08:16:27.005005+0000","last_undegraded":"2023-10-16T08:16:27.005005+0000","last_fullsized":"2023-10-16T08:16:27.005005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543245+0000","last_change":"2023-10-16T08:16:27.012879+0000","last_active":"2023-10-16T08:16:29.543245+0000","last_peered":"2023-10-16T08:16:29.543245+0000","last_clean":"2023-10-16T08:16:29.543245+0000","last_became_active":"2023-10-16T08:16:27.012466+0000","last_became_peered":"2023-10-16T08:16:27.012466+0000","last_unstale":"2023-10-16T08:16:29.543245+0000","last_undegraded":"2023-10-16T08:16:29.543245+0000","last_fullsized":"2023-10-16T08:16:29.543245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:16.486175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178093+0000","last_change":"2023-10-16T08:16:22.973193+0000","last_active":"2023-10-16T08:16:27.178093+0000","last_peered":"2023-10-16T08:16:27.178093+0000","last_clean":"2023-10-16T08:16:27.178093+0000","last_became_active":"2023-10-16T08:16:22.972472+0000","last_became_peered":"2023-10-16T08:16:22.972472+0000","last_unstale":"2023-10-16T08:16:27.178093+0000","last_undegraded":"2023-10-16T08:16:27.178093+0000","last_fullsized":"2023-10-16T08:16:27.178093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996389+0000","last_change":"2023-10-16T08:16:21.952668+0000","last_active":"2023-10-16T08:16:26.996389+0000","last_peered":"2023-10-16T08:16:26.996389+0000","last_clean":"2023-10-16T08:16:26.996389+0000","last_became_active":"2023-10-16T08:16:21.952006+0000","last_became_peered":"2023-10-16T08:16:21.952006+0000","last_unstale":"2023-10-16T08:16:26.996389+0000","last_undegraded":"2023-10-16T08:16:26.996389+0000","last_fullsized":"2023-10-16T08:16:26.996389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541359+0000","last_change":"2023-10-16T08:16:27.015675+0000","last_active":"2023-10-16T08:16:29.541359+0000","last_peered":"2023-10-16T08:16:29.541359+0000","last_clean":"2023-10-16T08:16:29.541359+0000","last_became_active":"2023-10-16T08:16:27.015279+0000","last_became_peered":"2023-10-16T08:16:27.015279+0000","last_unstale":"2023-10-16T08:16:29.541359+0000","last_undegraded":"2023-10-16T08:16:29.541359+0000","last_fullsized":"2023-10-16T08:16:29.541359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986776+0000","last_change":"2023-10-16T08:16:22.963194+0000","last_active":"2023-10-16T08:16:26.986776+0000","last_peered":"2023-10-16T08:16:26.986776+0000","last_clean":"2023-10-16T08:16:26.986776+0000","last_became_active":"2023-10-16T08:16:22.962589+0000","last_became_peered":"2023-10-16T08:16:22.962589+0000","last_unstale":"2023-10-16T08:16:26.986776+0000","last_undegraded":"2023-10-16T08:16:26.986776+0000","last_fullsized":"2023-10-16T08:16:26.986776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986872+0000","last_change":"2023-10-16T08:16:21.957192+0000","last_active":"2023-10-16T08:16:26.986872+0000","last_peered":"2023-10-16T08:16:26.986872+0000","last_clean":"2023-10-16T08:16:26.986872+0000","last_became_active":"2023-10-16T08:16:21.955564+0000","last_became_peered":"2023-10-16T08:16:21.955564+0000","last_unstale":"2023-10-16T08:16:26.986872+0000","last_undegraded":"2023-10-16T08:16:26.986872+0000","last_fullsized":"2023-10-16T08:16:26.986872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:58.308103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990800+0000","last_change":"2023-10-16T08:16:27.185040+0000","last_active":"2023-10-16T08:16:27.990800+0000","last_peered":"2023-10-16T08:16:27.990800+0000","last_clean":"2023-10-16T08:16:27.990800+0000","last_became_active":"2023-10-16T08:16:27.184448+0000","last_became_peered":"2023-10-16T08:16:27.184448+0000","last_unstale":"2023-10-16T08:16:27.990800+0000","last_undegraded":"2023-10-16T08:16:27.990800+0000","last_fullsized":"2023-10-16T08:16:27.990800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:59.852030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178532+0000","last_change":"2023-10-16T08:16:22.980031+0000","last_active":"2023-10-16T08:16:27.178532+0000","last_peered":"2023-10-16T08:16:27.178532+0000","last_clean":"2023-10-16T08:16:27.178532+0000","last_became_active":"2023-10-16T08:16:22.976708+0000","last_became_peered":"2023-10-16T08:16:22.976708+0000","last_unstale":"2023-10-16T08:16:27.178532+0000","last_undegraded":"2023-10-16T08:16:27.178532+0000","last_fullsized":"2023-10-16T08:16:27.178532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994242+0000","last_change":"2023-10-16T08:16:21.956986+0000","last_active":"2023-10-16T08:16:26.994242+0000","last_peered":"2023-10-16T08:16:26.994242+0000","last_clean":"2023-10-16T08:16:26.994242+0000","last_became_active":"2023-10-16T08:16:21.956307+0000","last_became_peered":"2023-10-16T08:16:21.956307+0000","last_unstale":"2023-10-16T08:16:26.994242+0000","last_undegraded":"2023-10-16T08:16:26.994242+0000","last_fullsized":"2023-10-16T08:16:26.994242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:26.142575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990801+0000","last_change":"2023-10-16T08:16:27.179150+0000","last_active":"2023-10-16T08:16:27.990801+0000","last_peered":"2023-10-16T08:16:27.990801+0000","last_clean":"2023-10-16T08:16:27.990801+0000","last_became_active":"2023-10-16T08:16:27.178649+0000","last_became_peered":"2023-10-16T08:16:27.178649+0000","last_unstale":"2023-10-16T08:16:27.990801+0000","last_undegraded":"2023-10-16T08:16:27.990801+0000","last_fullsized":"2023-10-16T08:16:27.990801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:45.567047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984308+0000","last_change":"2023-10-16T08:16:22.967505+0000","last_active":"2023-10-16T08:16:26.984308+0000","last_peered":"2023-10-16T08:16:26.984308+0000","last_clean":"2023-10-16T08:16:26.984308+0000","last_became_active":"2023-10-16T08:16:22.967214+0000","last_became_peered":"2023-10-16T08:16:22.967214+0000","last_unstale":"2023-10-16T08:16:26.984308+0000","last_undegraded":"2023-10-16T08:16:26.984308+0000","last_fullsized":"2023-10-16T08:16:26.984308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988187+0000","last_change":"2023-10-16T08:16:21.946248+0000","last_active":"2023-10-16T08:16:26.988187+0000","last_peered":"2023-10-16T08:16:26.988187+0000","last_clean":"2023-10-16T08:16:26.988187+0000","last_became_active":"2023-10-16T08:16:21.945958+0000","last_became_peered":"2023-10-16T08:16:21.945958+0000","last_unstale":"2023-10-16T08:16:26.988187+0000","last_undegraded":"2023-10-16T08:16:26.988187+0000","last_fullsized":"2023-10-16T08:16:26.988187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543632+0000","last_change":"2023-10-16T08:16:27.184504+0000","last_active":"2023-10-16T08:16:29.543632+0000","last_peered":"2023-10-16T08:16:29.543632+0000","last_clean":"2023-10-16T08:16:29.543632+0000","last_became_active":"2023-10-16T08:16:27.184029+0000","last_became_peered":"2023-10-16T08:16:27.184029+0000","last_unstale":"2023-10-16T08:16:29.543632+0000","last_undegraded":"2023-10-16T08:16:29.543632+0000","last_fullsized":"2023-10-16T08:16:29.543632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178733+0000","last_change":"2023-10-16T08:16:22.986066+0000","last_active":"2023-10-16T08:16:27.178733+0000","last_peered":"2023-10-16T08:16:27.178733+0000","last_clean":"2023-10-16T08:16:27.178733+0000","last_became_active":"2023-10-16T08:16:22.985791+0000","last_became_peered":"2023-10-16T08:16:22.985791+0000","last_unstale":"2023-10-16T08:16:27.178733+0000","last_undegraded":"2023-10-16T08:16:27.178733+0000","last_fullsized":"2023-10-16T08:16:27.178733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:38.000149+0000","last_change":"2023-10-16T08:16:21.955781+0000","last_active":"2023-10-16T08:16:38.000149+0000","last_peered":"2023-10-16T08:16:38.000149+0000","last_clean":"2023-10-16T08:16:38.000149+0000","last_became_active":"2023-10-16T08:16:21.955327+0000","last_became_peered":"2023-10-16T08:16:21.955327+0000","last_unstale":"2023-10-16T08:16:38.000149+0000","last_undegraded":"2023-10-16T08:16:38.000149+0000","last_fullsized":"2023-10-16T08:16:38.000149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987157+0000","last_change":"2023-10-16T08:16:27.007137+0000","last_active":"2023-10-16T08:16:27.987157+0000","last_peered":"2023-10-16T08:16:27.987157+0000","last_clean":"2023-10-16T08:16:27.987157+0000","last_became_active":"2023-10-16T08:16:27.006722+0000","last_became_peered":"2023-10-16T08:16:27.006722+0000","last_unstale":"2023-10-16T08:16:27.987157+0000","last_undegraded":"2023-10-16T08:16:27.987157+0000","last_fullsized":"2023-10-16T08:16:27.987157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004799+0000","last_change":"2023-10-16T08:16:22.976159+0000","last_active":"2023-10-16T08:16:27.004799+0000","last_peered":"2023-10-16T08:16:27.004799+0000","last_clean":"2023-10-16T08:16:27.004799+0000","last_became_active":"2023-10-16T08:16:22.975879+0000","last_became_peered":"2023-10-16T08:16:22.975879+0000","last_unstale":"2023-10-16T08:16:27.004799+0000","last_undegraded":"2023-10-16T08:16:27.004799+0000","last_fullsized":"2023-10-16T08:16:27.004799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:04.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013901+0000","last_change":"2023-10-16T08:16:21.961500+0000","last_active":"2023-10-16T08:16:27.013901+0000","last_peered":"2023-10-16T08:16:27.013901+0000","last_clean":"2023-10-16T08:16:27.013901+0000","last_became_active":"2023-10-16T08:16:21.959870+0000","last_became_peered":"2023-10-16T08:16:21.959870+0000","last_unstale":"2023-10-16T08:16:27.013901+0000","last_undegraded":"2023-10-16T08:16:27.013901+0000","last_fullsized":"2023-10-16T08:16:27.013901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543647+0000","last_change":"2023-10-16T08:16:27.018940+0000","last_active":"2023-10-16T08:16:29.543647+0000","last_peered":"2023-10-16T08:16:29.543647+0000","last_clean":"2023-10-16T08:16:29.543647+0000","last_became_active":"2023-10-16T08:16:27.018428+0000","last_became_peered":"2023-10-16T08:16:27.018428+0000","last_unstale":"2023-10-16T08:16:29.543647+0000","last_undegraded":"2023-10-16T08:16:29.543647+0000","last_fullsized":"2023-10-16T08:16:29.543647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988140+0000","last_change":"2023-10-16T08:16:22.974536+0000","last_active":"2023-10-16T08:16:26.988140+0000","last_peered":"2023-10-16T08:16:26.988140+0000","last_clean":"2023-10-16T08:16:26.988140+0000","last_became_active":"2023-10-16T08:16:22.973582+0000","last_became_peered":"2023-10-16T08:16:22.973582+0000","last_unstale":"2023-10-16T08:16:26.988140+0000","last_undegraded":"2023-10-16T08:16:26.988140+0000","last_fullsized":"2023-10-16T08:16:26.988140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995774+0000","last_change":"2023-10-16T08:16:21.960308+0000","last_active":"2023-10-16T08:16:26.995774+0000","last_peered":"2023-10-16T08:16:26.995774+0000","last_clean":"2023-10-16T08:16:26.995774+0000","last_became_active":"2023-10-16T08:16:21.959328+0000","last_became_peered":"2023-10-16T08:16:21.959328+0000","last_unstale":"2023-10-16T08:16:26.995774+0000","last_undegraded":"2023-10-16T08:16:26.995774+0000","last_fullsized":"2023-10-16T08:16:26.995774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:15.899771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989677+0000","last_change":"2023-10-16T08:16:27.007716+0000","last_active":"2023-10-16T08:16:27.989677+0000","last_peered":"2023-10-16T08:16:27.989677+0000","last_clean":"2023-10-16T08:16:27.989677+0000","last_became_active":"2023-10-16T08:16:27.006885+0000","last_became_peered":"2023-10-16T08:16:27.006885+0000","last_unstale":"2023-10-16T08:16:27.989677+0000","last_undegraded":"2023-10-16T08:16:27.989677+0000","last_fullsized":"2023-10-16T08:16:27.989677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:18.625895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995685+0000","last_change":"2023-10-16T08:16:22.990790+0000","last_active":"2023-10-16T08:16:26.995685+0000","last_peered":"2023-10-16T08:16:26.995685+0000","last_clean":"2023-10-16T08:16:26.995685+0000","last_became_active":"2023-10-16T08:16:22.987169+0000","last_became_peered":"2023-10-16T08:16:22.987169+0000","last_unstale":"2023-10-16T08:16:26.995685+0000","last_undegraded":"2023-10-16T08:16:26.995685+0000","last_fullsized":"2023-10-16T08:16:26.995685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988440+0000","last_change":"2023-10-16T08:16:21.956681+0000","last_active":"2023-10-16T08:16:26.988440+0000","last_peered":"2023-10-16T08:16:26.988440+0000","last_clean":"2023-10-16T08:16:26.988440+0000","last_became_active":"2023-10-16T08:16:21.956318+0000","last_became_peered":"2023-10-16T08:16:21.956318+0000","last_unstale":"2023-10-16T08:16:26.988440+0000","last_undegraded":"2023-10-16T08:16:26.988440+0000","last_fullsized":"2023-10-16T08:16:26.988440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989667+0000","last_change":"2023-10-16T08:16:27.181126+0000","last_active":"2023-10-16T08:16:27.989667+0000","last_peered":"2023-10-16T08:16:27.989667+0000","last_clean":"2023-10-16T08:16:27.989667+0000","last_became_active":"2023-10-16T08:16:27.180722+0000","last_became_peered":"2023-10-16T08:16:27.180722+0000","last_unstale":"2023-10-16T08:16:27.989667+0000","last_undegraded":"2023-10-16T08:16:27.989667+0000","last_fullsized":"2023-10-16T08:16:27.989667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:39:03.047765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541176+0000","last_change":"2023-10-16T08:16:27.183947+0000","last_active":"2023-10-16T08:16:29.541176+0000","last_peered":"2023-10-16T08:16:29.541176+0000","last_clean":"2023-10-16T08:16:29.541176+0000","last_became_active":"2023-10-16T08:16:27.183487+0000","last_became_peered":"2023-10-16T08:16:27.183487+0000","last_unstale":"2023-10-16T08:16:29.541176+0000","last_undegraded":"2023-10-16T08:16:29.541176+0000","last_fullsized":"2023-10-16T08:16:29.541176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009520+0000","last_change":"2023-10-16T08:16:22.958791+0000","last_active":"2023-10-16T08:16:27.009520+0000","last_peered":"2023-10-16T08:16:27.009520+0000","last_clean":"2023-10-16T08:16:27.009520+0000","last_became_active":"2023-10-16T08:16:22.958513+0000","last_became_peered":"2023-10-16T08:16:22.958513+0000","last_unstale":"2023-10-16T08:16:27.009520+0000","last_undegraded":"2023-10-16T08:16:27.009520+0000","last_fullsized":"2023-10-16T08:16:27.009520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994423+0000","last_change":"2023-10-16T08:16:21.954384+0000","last_active":"2023-10-16T08:16:26.994423+0000","last_peered":"2023-10-16T08:16:26.994423+0000","last_clean":"2023-10-16T08:16:26.994423+0000","last_became_active":"2023-10-16T08:16:21.953995+0000","last_became_peered":"2023-10-16T08:16:21.953995+0000","last_unstale":"2023-10-16T08:16:26.994423+0000","last_undegraded":"2023-10-16T08:16:26.994423+0000","last_fullsized":"2023-10-16T08:16:26.994423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:34.686144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.992231+0000","last_change":"2023-10-16T08:16:27.008716+0000","last_active":"2023-10-16T08:16:27.992231+0000","last_peered":"2023-10-16T08:16:27.992231+0000","last_clean":"2023-10-16T08:16:27.992231+0000","last_became_active":"2023-10-16T08:16:27.007942+0000","last_became_peered":"2023-10-16T08:16:27.007942+0000","last_unstale":"2023-10-16T08:16:27.992231+0000","last_undegraded":"2023-10-16T08:16:27.992231+0000","last_fullsized":"2023-10-16T08:16:27.992231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986408+0000","last_change":"2023-10-16T08:16:22.969567+0000","last_active":"2023-10-16T08:16:26.986408+0000","last_peered":"2023-10-16T08:16:26.986408+0000","last_clean":"2023-10-16T08:16:26.986408+0000","last_became_active":"2023-10-16T08:16:22.969172+0000","last_became_peered":"2023-10-16T08:16:22.969172+0000","last_unstale":"2023-10-16T08:16:26.986408+0000","last_undegraded":"2023-10-16T08:16:26.986408+0000","last_fullsized":"2023-10-16T08:16:26.986408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986465+0000","last_change":"2023-10-16T08:16:21.956899+0000","last_active":"2023-10-16T08:16:26.986465+0000","last_peered":"2023-10-16T08:16:26.986465+0000","last_clean":"2023-10-16T08:16:26.986465+0000","last_became_active":"2023-10-16T08:16:21.956446+0000","last_became_peered":"2023-10-16T08:16:21.956446+0000","last_unstale":"2023-10-16T08:16:26.986465+0000","last_undegraded":"2023-10-16T08:16:26.986465+0000","last_fullsized":"2023-10-16T08:16:26.986465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987260+0000","last_change":"2023-10-16T08:16:21.955151+0000","last_active":"2023-10-16T08:16:26.987260+0000","last_peered":"2023-10-16T08:16:26.987260+0000","last_clean":"2023-10-16T08:16:26.987260+0000","last_became_active":"2023-10-16T08:16:21.953524+0000","last_became_peered":"2023-10-16T08:16:21.953524+0000","last_unstale":"2023-10-16T08:16:26.987260+0000","last_undegraded":"2023-10-16T08:16:26.987260+0000","last_fullsized":"2023-10-16T08:16:26.987260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988088+0000","last_change":"2023-10-16T08:16:22.965640+0000","last_active":"2023-10-16T08:16:26.988088+0000","last_peered":"2023-10-16T08:16:26.988088+0000","last_clean":"2023-10-16T08:16:26.988088+0000","last_became_active":"2023-10-16T08:16:22.963142+0000","last_became_peered":"2023-10-16T08:16:22.963142+0000","last_unstale":"2023-10-16T08:16:26.988088+0000","last_undegraded":"2023-10-16T08:16:26.988088+0000","last_fullsized":"2023-10-16T08:16:26.988088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.540964+0000","last_change":"2023-10-16T08:16:27.020258+0000","last_active":"2023-10-16T08:16:29.540964+0000","last_peered":"2023-10-16T08:16:29.540964+0000","last_clean":"2023-10-16T08:16:29.540964+0000","last_became_active":"2023-10-16T08:16:27.019846+0000","last_became_peered":"2023-10-16T08:16:27.019846+0000","last_unstale":"2023-10-16T08:16:29.540964+0000","last_undegraded":"2023-10-16T08:16:29.540964+0000","last_fullsized":"2023-10-16T08:16:29.540964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:47.301298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990459+0000","last_change":"2023-10-16T08:16:21.959137+0000","last_active":"2023-10-16T08:16:26.990459+0000","last_peered":"2023-10-16T08:16:26.990459+0000","last_clean":"2023-10-16T08:16:26.990459+0000","last_became_active":"2023-10-16T08:16:21.958785+0000","last_became_peered":"2023-10-16T08:16:21.958785+0000","last_unstale":"2023-10-16T08:16:26.990459+0000","last_undegraded":"2023-10-16T08:16:26.990459+0000","last_fullsized":"2023-10-16T08:16:26.990459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.999013+0000","last_change":"2023-10-16T08:16:22.961411+0000","last_active":"2023-10-16T08:16:26.999013+0000","last_peered":"2023-10-16T08:16:26.999013+0000","last_clean":"2023-10-16T08:16:26.999013+0000","last_became_active":"2023-10-16T08:16:22.961004+0000","last_became_peered":"2023-10-16T08:16:22.961004+0000","last_unstale":"2023-10-16T08:16:26.999013+0000","last_undegraded":"2023-10-16T08:16:26.999013+0000","last_fullsized":"2023-10-16T08:16:26.999013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:11.319467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990247+0000","last_change":"2023-10-16T08:16:27.014631+0000","last_active":"2023-10-16T08:16:27.990247+0000","last_peered":"2023-10-16T08:16:27.990247+0000","last_clean":"2023-10-16T08:16:27.990247+0000","last_became_active":"2023-10-16T08:16:27.014246+0000","last_became_peered":"2023-10-16T08:16:27.014246+0000","last_unstale":"2023-10-16T08:16:27.990247+0000","last_undegraded":"2023-10-16T08:16:27.990247+0000","last_fullsized":"2023-10-16T08:16:27.990247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:43.540909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986773+0000","last_change":"2023-10-16T08:16:22.971903+0000","last_active":"2023-10-16T08:16:26.986773+0000","last_peered":"2023-10-16T08:16:26.986773+0000","last_clean":"2023-10-16T08:16:26.986773+0000","last_became_active":"2023-10-16T08:16:22.971522+0000","last_became_peered":"2023-10-16T08:16:22.971522+0000","last_unstale":"2023-10-16T08:16:26.986773+0000","last_undegraded":"2023-10-16T08:16:26.986773+0000","last_fullsized":"2023-10-16T08:16:26.986773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996570+0000","last_change":"2023-10-16T08:16:22.970072+0000","last_active":"2023-10-16T08:16:26.996570+0000","last_peered":"2023-10-16T08:16:26.996570+0000","last_clean":"2023-10-16T08:16:26.996570+0000","last_became_active":"2023-10-16T08:16:22.969751+0000","last_became_peered":"2023-10-16T08:16:22.969751+0000","last_unstale":"2023-10-16T08:16:26.996570+0000","last_undegraded":"2023-10-16T08:16:26.996570+0000","last_fullsized":"2023-10-16T08:16:26.996570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994134+0000","last_change":"2023-10-16T08:16:22.995816+0000","last_active":"2023-10-16T08:16:26.994134+0000","last_peered":"2023-10-16T08:16:26.994134+0000","last_clean":"2023-10-16T08:16:26.994134+0000","last_became_active":"2023-10-16T08:16:22.990930+0000","last_became_peered":"2023-10-16T08:16:22.990930+0000","last_unstale":"2023-10-16T08:16:26.994134+0000","last_undegraded":"2023-10-16T08:16:26.994134+0000","last_fullsized":"2023-10-16T08:16:26.994134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987998+0000","last_change":"2023-10-16T08:16:22.972789+0000","last_active":"2023-10-16T08:16:26.987998+0000","last_peered":"2023-10-16T08:16:26.987998+0000","last_clean":"2023-10-16T08:16:26.987998+0000","last_became_active":"2023-10-16T08:16:22.972473+0000","last_became_peered":"2023-10-16T08:16:22.972473+0000","last_unstale":"2023-10-16T08:16:26.987998+0000","last_undegraded":"2023-10-16T08:16:26.987998+0000","last_fullsized":"2023-10-16T08:16:26.987998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985911+0000","last_change":"2023-10-16T08:16:22.971086+0000","last_active":"2023-10-16T08:16:26.985911+0000","last_peered":"2023-10-16T08:16:26.985911+0000","last_clean":"2023-10-16T08:16:26.985911+0000","last_became_active":"2023-10-16T08:16:22.962675+0000","last_became_peered":"2023-10-16T08:16:22.962675+0000","last_unstale":"2023-10-16T08:16:26.985911+0000","last_undegraded":"2023-10-16T08:16:26.985911+0000","last_fullsized":"2023-10-16T08:16:26.985911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013810+0000","last_change":"2023-10-16T08:16:22.962278+0000","last_active":"2023-10-16T08:16:27.013810+0000","last_peered":"2023-10-16T08:16:27.013810+0000","last_clean":"2023-10-16T08:16:27.013810+0000","last_became_active":"2023-10-16T08:16:22.962006+0000","last_became_peered":"2023-10-16T08:16:22.962006+0000","last_unstale":"2023-10-16T08:16:27.013810+0000","last_undegraded":"2023-10-16T08:16:27.013810+0000","last_fullsized":"2023-10-16T08:16:27.013810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975064+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.974692+0000","last_became_peered":"2023-10-16T08:16:22.974692+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987037+0000","last_change":"2023-10-16T08:16:22.961194+0000","last_active":"2023-10-16T08:16:26.987037+0000","last_peered":"2023-10-16T08:16:26.987037+0000","last_clean":"2023-10-16T08:16:26.987037+0000","last_became_active":"2023-10-16T08:16:22.960884+0000","last_became_peered":"2023-10-16T08:16:22.960884+0000","last_unstale":"2023-10-16T08:16:26.987037+0000","last_undegraded":"2023-10-16T08:16:26.987037+0000","last_fullsized":"2023-10-16T08:16:26.987037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994288+0000","last_change":"2023-10-16T08:16:22.983850+0000","last_active":"2023-10-16T08:16:26.994288+0000","last_peered":"2023-10-16T08:16:26.994288+0000","last_clean":"2023-10-16T08:16:26.994288+0000","last_became_active":"2023-10-16T08:16:22.978686+0000","last_became_peered":"2023-10-16T08:16:22.978686+0000","last_unstale":"2023-10-16T08:16:26.994288+0000","last_undegraded":"2023-10-16T08:16:26.994288+0000","last_fullsized":"2023-10-16T08:16:26.994288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986300+0000","last_change":"2023-10-16T08:16:22.957405+0000","last_active":"2023-10-16T08:16:26.986300+0000","last_peered":"2023-10-16T08:16:26.986300+0000","last_clean":"2023-10-16T08:16:26.986300+0000","last_became_active":"2023-10-16T08:16:22.957105+0000","last_became_peered":"2023-10-16T08:16:22.957105+0000","last_unstale":"2023-10-16T08:16:26.986300+0000","last_undegraded":"2023-10-16T08:16:26.986300+0000","last_fullsized":"2023-10-16T08:16:26.986300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994045+0000","last_change":"2023-10-16T08:16:22.985035+0000","last_active":"2023-10-16T08:16:26.994045+0000","last_peered":"2023-10-16T08:16:26.994045+0000","last_clean":"2023-10-16T08:16:26.994045+0000","last_became_active":"2023-10-16T08:16:22.984750+0000","last_became_peered":"2023-10-16T08:16:22.984750+0000","last_unstale":"2023-10-16T08:16:26.994045+0000","last_undegraded":"2023-10-16T08:16:26.994045+0000","last_fullsized":"2023-10-16T08:16:26.994045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987250+0000","last_change":"2023-10-16T08:16:22.976322+0000","last_active":"2023-10-16T08:16:26.987250+0000","last_peered":"2023-10-16T08:16:26.987250+0000","last_clean":"2023-10-16T08:16:26.987250+0000","last_became_active":"2023-10-16T08:16:22.976064+0000","last_became_peered":"2023-10-16T08:16:22.976064+0000","last_unstale":"2023-10-16T08:16:26.987250+0000","last_undegraded":"2023-10-16T08:16:26.987250+0000","last_fullsized":"2023-10-16T08:16:26.987250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987111+0000","last_change":"2023-10-16T08:16:22.965222+0000","last_active":"2023-10-16T08:16:26.987111+0000","last_peered":"2023-10-16T08:16:26.987111+0000","last_clean":"2023-10-16T08:16:26.987111+0000","last_became_active":"2023-10-16T08:16:22.964839+0000","last_became_peered":"2023-10-16T08:16:22.964839+0000","last_unstale":"2023-10-16T08:16:26.987111+0000","last_undegraded":"2023-10-16T08:16:26.987111+0000","last_fullsized":"2023-10-16T08:16:26.987111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.978846+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.975261+0000","last_became_peered":"2023-10-16T08:16:22.975261+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996150+0000","last_change":"2023-10-16T08:16:22.973907+0000","last_active":"2023-10-16T08:16:26.996150+0000","last_peered":"2023-10-16T08:16:26.996150+0000","last_clean":"2023-10-16T08:16:26.996150+0000","last_became_active":"2023-10-16T08:16:22.973609+0000","last_became_peered":"2023-10-16T08:16:22.973609+0000","last_unstale":"2023-10-16T08:16:26.996150+0000","last_undegraded":"2023-10-16T08:16:26.996150+0000","last_fullsized":"2023-10-16T08:16:26.996150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994986+0000","last_change":"2023-10-16T08:16:22.995822+0000","last_active":"2023-10-16T08:16:26.994986+0000","last_peered":"2023-10-16T08:16:26.994986+0000","last_clean":"2023-10-16T08:16:26.994986+0000","last_became_active":"2023-10-16T08:16:22.995473+0000","last_became_peered":"2023-10-16T08:16:22.995473+0000","last_unstale":"2023-10-16T08:16:26.994986+0000","last_undegraded":"2023-10-16T08:16:26.994986+0000","last_fullsized":"2023-10-16T08:16:26.994986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994188+0000","last_change":"2023-10-16T08:16:22.975976+0000","last_active":"2023-10-16T08:16:26.994188+0000","last_peered":"2023-10-16T08:16:26.994188+0000","last_clean":"2023-10-16T08:16:26.994188+0000","last_became_active":"2023-10-16T08:16:22.975600+0000","last_became_peered":"2023-10-16T08:16:22.975600+0000","last_unstale":"2023-10-16T08:16:26.994188+0000","last_undegraded":"2023-10-16T08:16:26.994188+0000","last_fullsized":"2023-10-16T08:16:26.994188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009586+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.009586+0000","last_peered":"2023-10-16T08:16:27.009586+0000","last_clean":"2023-10-16T08:16:27.009586+0000","last_became_active":"2023-10-16T08:16:22.954170+0000","last_became_peered":"2023-10-16T08:16:22.954170+0000","last_unstale":"2023-10-16T08:16:27.009586+0000","last_undegraded":"2023-10-16T08:16:27.009586+0000","last_fullsized":"2023-10-16T08:16:27.009586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984253+0000","last_change":"2023-10-16T08:16:22.959029+0000","last_active":"2023-10-16T08:16:26.984253+0000","last_peered":"2023-10-16T08:16:26.984253+0000","last_clean":"2023-10-16T08:16:26.984253+0000","last_became_active":"2023-10-16T08:16:22.958713+0000","last_became_peered":"2023-10-16T08:16:22.958713+0000","last_unstale":"2023-10-16T08:16:26.984253+0000","last_undegraded":"2023-10-16T08:16:26.984253+0000","last_fullsized":"2023-10-16T08:16:26.984253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:47.145986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990281+0000","last_change":"2023-10-16T08:16:22.973255+0000","last_active":"2023-10-16T08:16:26.990281+0000","last_peered":"2023-10-16T08:16:26.990281+0000","last_clean":"2023-10-16T08:16:26.990281+0000","last_became_active":"2023-10-16T08:16:22.972582+0000","last_became_peered":"2023-10-16T08:16:22.972582+0000","last_unstale":"2023-10-16T08:16:26.990281+0000","last_undegraded":"2023-10-16T08:16:26.990281+0000","last_fullsized":"2023-10-16T08:16:26.990281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004704+0000","last_change":"2023-10-16T08:16:22.969190+0000","last_active":"2023-10-16T08:16:27.004704+0000","last_peered":"2023-10-16T08:16:27.004704+0000","last_clean":"2023-10-16T08:16:27.004704+0000","last_became_active":"2023-10-16T08:16:22.968880+0000","last_became_peered":"2023-10-16T08:16:22.968880+0000","last_unstale":"2023-10-16T08:16:27.004704+0000","last_undegraded":"2023-10-16T08:16:27.004704+0000","last_fullsized":"2023-10-16T08:16:27.004704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987745+0000","last_change":"2023-10-16T08:16:22.975463+0000","last_active":"2023-10-16T08:16:26.987745+0000","last_peered":"2023-10-16T08:16:26.987745+0000","last_clean":"2023-10-16T08:16:26.987745+0000","last_became_active":"2023-10-16T08:16:22.975080+0000","last_became_peered":"2023-10-16T08:16:22.975080+0000","last_unstale":"2023-10-16T08:16:26.987745+0000","last_undegraded":"2023-10-16T08:16:26.987745+0000","last_fullsized":"2023-10-16T08:16:26.987745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975707+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.975429+0000","last_became_peered":"2023-10-16T08:16:22.975429+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179066+0000","last_change":"2023-10-16T08:16:22.977921+0000","last_active":"2023-10-16T08:16:27.179066+0000","last_peered":"2023-10-16T08:16:27.179066+0000","last_clean":"2023-10-16T08:16:27.179066+0000","last_became_active":"2023-10-16T08:16:22.974938+0000","last_became_peered":"2023-10-16T08:16:22.974938+0000","last_unstale":"2023-10-16T08:16:27.179066+0000","last_undegraded":"2023-10-16T08:16:27.179066+0000","last_fullsized":"2023-10-16T08:16:27.179066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:35.197545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986251+0000","last_change":"2023-10-16T08:16:22.972026+0000","last_active":"2023-10-16T08:16:26.986251+0000","last_peered":"2023-10-16T08:16:26.986251+0000","last_clean":"2023-10-16T08:16:26.986251+0000","last_became_active":"2023-10-16T08:16:22.971655+0000","last_became_peered":"2023-10-16T08:16:22.971655+0000","last_unstale":"2023-10-16T08:16:26.986251+0000","last_undegraded":"2023-10-16T08:16:26.986251+0000","last_fullsized":"2023-10-16T08:16:26.986251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178078+0000","last_change":"2023-10-16T08:16:22.973377+0000","last_active":"2023-10-16T08:16:27.178078+0000","last_peered":"2023-10-16T08:16:27.178078+0000","last_clean":"2023-10-16T08:16:27.178078+0000","last_became_active":"2023-10-16T08:16:22.972888+0000","last_became_peered":"2023-10-16T08:16:22.972888+0000","last_unstale":"2023-10-16T08:16:27.178078+0000","last_undegraded":"2023-10-16T08:16:27.178078+0000","last_fullsized":"2023-10-16T08:16:27.178078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994140+0000","last_change":"2023-10-16T08:16:22.976999+0000","last_active":"2023-10-16T08:16:26.994140+0000","last_peered":"2023-10-16T08:16:26.994140+0000","last_clean":"2023-10-16T08:16:26.994140+0000","last_became_active":"2023-10-16T08:16:22.974943+0000","last_became_peered":"2023-10-16T08:16:22.974943+0000","last_unstale":"2023-10-16T08:16:26.994140+0000","last_undegraded":"2023-10-16T08:16:26.994140+0000","last_fullsized":"2023-10-16T08:16:26.994140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178298+0000","last_change":"2023-10-16T08:16:22.979562+0000","last_active":"2023-10-16T08:16:27.178298+0000","last_peered":"2023-10-16T08:16:27.178298+0000","last_clean":"2023-10-16T08:16:27.178298+0000","last_became_active":"2023-10-16T08:16:22.975468+0000","last_became_peered":"2023-10-16T08:16:22.975468+0000","last_unstale":"2023-10-16T08:16:27.178298+0000","last_undegraded":"2023-10-16T08:16:27.178298+0000","last_fullsized":"2023-10-16T08:16:27.178298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994788+0000","last_change":"2023-10-16T08:16:22.991415+0000","last_active":"2023-10-16T08:16:26.994788+0000","last_peered":"2023-10-16T08:16:26.994788+0000","last_clean":"2023-10-16T08:16:26.994788+0000","last_became_active":"2023-10-16T08:16:22.990410+0000","last_became_peered":"2023-10-16T08:16:22.990410+0000","last_unstale":"2023-10-16T08:16:26.994788+0000","last_undegraded":"2023-10-16T08:16:26.994788+0000","last_fullsized":"2023-10-16T08:16:26.994788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987345+0000","last_change":"2023-10-16T08:16:22.957927+0000","last_active":"2023-10-16T08:16:26.987345+0000","last_peered":"2023-10-16T08:16:26.987345+0000","last_clean":"2023-10-16T08:16:26.987345+0000","last_became_active":"2023-10-16T08:16:22.956909+0000","last_became_peered":"2023-10-16T08:16:22.956909+0000","last_unstale":"2023-10-16T08:16:26.987345+0000","last_undegraded":"2023-10-16T08:16:26.987345+0000","last_fullsized":"2023-10-16T08:16:26.987345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014243+0000","last_change":"2023-10-16T08:16:22.963419+0000","last_active":"2023-10-16T08:16:27.014243+0000","last_peered":"2023-10-16T08:16:27.014243+0000","last_clean":"2023-10-16T08:16:27.014243+0000","last_became_active":"2023-10-16T08:16:22.962692+0000","last_became_peered":"2023-10-16T08:16:22.962692+0000","last_unstale":"2023-10-16T08:16:27.014243+0000","last_undegraded":"2023-10-16T08:16:27.014243+0000","last_fullsized":"2023-10-16T08:16:27.014243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178991+0000","last_change":"2023-10-16T08:16:22.976621+0000","last_active":"2023-10-16T08:16:27.178991+0000","last_peered":"2023-10-16T08:16:27.178991+0000","last_clean":"2023-10-16T08:16:27.178991+0000","last_became_active":"2023-10-16T08:16:22.975391+0000","last_became_peered":"2023-10-16T08:16:22.975391+0000","last_unstale":"2023-10-16T08:16:27.178991+0000","last_undegraded":"2023-10-16T08:16:27.178991+0000","last_fullsized":"2023-10-16T08:16:27.178991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:16:48.319 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T08:16:48.320 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-16T08:16:48.320 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-16T08:16:48.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T08:16:48.321 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-16T08:16:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 Processor -- start 2023-10-16T08:16:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 -- start start 2023-10-16T08:16:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c070c40 0x7f597c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f597c0b4020 0x7f597c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c15be60 con 0x7f597c0b0620 2023-10-16T08:16:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597c15b3b0 con 0x7f597c0b4020 2023-10-16T08:16:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f5981d3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c0b0a90 con 0x7f597c070c40 2023-10-16T08:16:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f597c0b4020 0x7f597c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c070c40 0x7f597c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f597c0b4020 0x7f597c0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41776/0 (socket says 172.21.15.32:41776) 2023-10-16T08:16:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 -- 172.21.15.32:0/3737919164 learned_addr learned my addr 172.21.15.32:0/3737919164 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 -- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c070c40 msgr2=0x7f597c15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 --2- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c070c40 0x7f597c15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 -- 172.21.15.32:0/3737919164 >> v1:172.21.15.186:6789/0 conn(0x7f597c0b0620 legacy=0x7f597c15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597affd640 1 -- 172.21.15.32:0/3737919164 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c15c1f0 con 0x7f597c0b4020 2023-10-16T08:16:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.498+0000 7f597b7fe640 1 --2- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c070c40 0x7f597c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f597affd640 1 --2- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c0b4020 0x7f597c0b00e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5970009870 tx=0x7f597001c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8a7761f91e0a55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f597a7fc640 1 -- 172.21.15.32:0/3737919164 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700aa610 con 0x7f597c0b4020 2023-10-16T08:16:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f597a7fc640 1 -- 172.21.15.32:0/3737919164 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59700a6040 con 0x7f597c0b4020 2023-10-16T08:16:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f597a7fc640 1 -- 172.21.15.32:0/3737919164 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700aa990 con 0x7f597c0b4020 2023-10-16T08:16:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 -- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c0b4020 msgr2=0x7f597c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 --2- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c0b4020 0x7f597c0b00e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5970009870 tx=0x7f597001c4b0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 -- 172.21.15.32:0/3737919164 shutdown_connections 2023-10-16T08:16:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 --2- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c0b4020 0x7f597c0b00e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 --2- 172.21.15.32:0/3737919164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c070c40 0x7f597c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 -- 172.21.15.32:0/3737919164 >> 172.21.15.32:0/3737919164 conn(0x7f597c06c190 msgr2=0x7f597c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:48.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 -- 172.21.15.32:0/3737919164 shutdown_connections 2023-10-16T08:16:48.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.502+0000 7f5981d3e640 1 -- 172.21.15.32:0/3737919164 wait complete. 2023-10-16T08:16:48.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 Processor -- start 2023-10-16T08:16:48.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 -- start start 2023-10-16T08:16:48.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c070c40 0x7f597c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c0b0620 0x7f597c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 0x7f597c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c1550c0 con 0x7f597c0b4020 2023-10-16T08:16:48.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c152ef0 con 0x7f597c070c40 2023-10-16T08:16:48.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5981d3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c1531d0 con 0x7f597c0b0620 2023-10-16T08:16:48.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 0x7f597c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 0x7f597c1549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58976/0 (socket says 172.21.15.32:58976) 2023-10-16T08:16:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 -- 172.21.15.32:0/4176671947 learned_addr learned my addr 172.21.15.32:0/4176671947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597affd640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c0b0620 0x7f597c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597b7fe640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c070c40 0x7f597c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 -- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c0b0620 msgr2=0x7f597c14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c0b0620 0x7f597c14b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 -- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c070c40 msgr2=0x7f597c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c070c40 0x7f597c1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 -- 172.21.15.32:0/4176671947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c009420 con 0x7f597c0b4020 2023-10-16T08:16:48.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f597bfff640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 0x7f597c1549e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f597400c6c0 tx=0x7f597400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.506+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974003b80 con 0x7f597c0b4020 2023-10-16T08:16:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5974003ce0 con 0x7f597c0b4020 2023-10-16T08:16:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59700094d0 con 0x7f597c0b4020 2023-10-16T08:16:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740052f0 con 0x7f597c0b4020 2023-10-16T08:16:48.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c14c160 con 0x7f597c0b4020 2023-10-16T08:16:48.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c074010 con 0x7f597c0b4020 2023-10-16T08:16:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5974005450 con 0x7f597c0b4020 2023-10-16T08:16:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5963fff640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5974083f30 con 0x7f597c0b4020 2023-10-16T08:16:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.510+0000 7f597b7fe640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.514+0000 7f597b7fe640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f596c009c80 tx=0x7f596c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.514+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59740510b0 con 0x7f597c0b4020 2023-10-16T08:16:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.710+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f597c0ccf40 con 0x7f597c0b4020 2023-10-16T08:16:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.710+0000 7f5963fff640 1 -- 172.21.15.32:0/4176671947 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164430 (secure 0 0 0) 0x7f59740546d0 con 0x7f597c0b4020 2023-10-16T08:16:48.719 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 msgr2=0x7f5968068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f596c009c80 tx=0x7f596c008040 comp rx=0 tx=0).stop 2023-10-16T08:16:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 msgr2=0x7f597c1549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 0x7f597c1549e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f597400c6c0 tx=0x7f597400cb80 comp rx=0 tx=0).stop 2023-10-16T08:16:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 shutdown_connections 2023-10-16T08:16:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f597c0b4020 0x7f597c1549e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f597c0b0620 0x7f597c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 --2- 172.21.15.32:0/4176671947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f597c070c40 0x7f597c1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 >> 172.21.15.32:0/4176671947 conn(0x7f597c06c190 msgr2=0x7f597c0afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 shutdown_connections 2023-10-16T08:16:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.714+0000 7f5981d3e640 1 -- 172.21.15.32:0/4176671947 wait complete. 2023-10-16T08:16:48.739 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":255423679},{"type":"v1","addr":"172.21.15.186:6833","nonce":255423679}]},"active_addr":"172.21.15.186:6833/255423679","active_change":"2023-10-16T08:16:01.996208+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4122,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":42245250}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":2044693624}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":25087867}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.186:0","nonce":3474634111}]}]} 2023-10-16T08:16:48.743 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-16T08:16:48.743 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-16T08:16:48.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:48.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 Processor -- start 2023-10-16T08:16:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 -- start start 2023-10-16T08:16:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c15cb90 0x7f965c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f965c0d3010 0x7f965c0d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c0ce770 con 0x7f965c0d1400 2023-10-16T08:16:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965c0d1890 con 0x7f965c0d3010 2023-10-16T08:16:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f966286b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0d0600 con 0x7f965c15cb90 2023-10-16T08:16:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f965c0d3010 0x7f965c0d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f965c0d3010 0x7f965c0d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58980/0 (socket says 172.21.15.32:58980) 2023-10-16T08:16:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 -- 172.21.15.32:0/2368170760 learned_addr learned my addr 172.21.15.32:0/2368170760 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 -- 172.21.15.32:0/2368170760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c15cb90 msgr2=0x7f965c0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965bfff640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c15cb90 0x7f965c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c15cb90 0x7f965c0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 -- 172.21.15.32:0/2368170760 >> v1:172.21.15.186:6789/0 conn(0x7f965c0d1400 legacy=0x7f965c0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965b7fe640 1 -- 172.21.15.32:0/2368170760 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0c6450 con 0x7f965c0d3010 2023-10-16T08:16:48.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.882+0000 7f965bfff640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c15cb90 0x7f965c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f965b7fe640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c0d0ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9650009870 tx=0x7f965001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68c3cff27c37adc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f965affd640 1 -- 172.21.15.32:0/2368170760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500ab600 con 0x7f965c0d3010 2023-10-16T08:16:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f965affd640 1 -- 172.21.15.32:0/2368170760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96500ab760 con 0x7f965c0d3010 2023-10-16T08:16:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f965affd640 1 -- 172.21.15.32:0/2368170760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500abaf0 con 0x7f965c0d3010 2023-10-16T08:16:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 -- 172.21.15.32:0/2368170760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 msgr2=0x7f965c0d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c0d0ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9650009870 tx=0x7f965001c440 comp rx=0 tx=0).stop 2023-10-16T08:16:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 -- 172.21.15.32:0/2368170760 shutdown_connections 2023-10-16T08:16:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c0d0ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 --2- 172.21.15.32:0/2368170760 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c15cb90 0x7f965c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 -- 172.21.15.32:0/2368170760 >> 172.21.15.32:0/2368170760 conn(0x7f965c09bc50 msgr2=0x7f965c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 -- 172.21.15.32:0/2368170760 shutdown_connections 2023-10-16T08:16:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.886+0000 7f966286b640 1 -- 172.21.15.32:0/2368170760 wait complete. 2023-10-16T08:16:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 Processor -- start 2023-10-16T08:16:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 -- start start 2023-10-16T08:16:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c0d1400 0x7f965c1663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965c15cb90 0x7f965c0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c1670b0 con 0x7f965c0d3010 2023-10-16T08:16:48.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c158040 con 0x7f965c15cb90 2023-10-16T08:16:48.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0cb860 con 0x7f965c0d1400 2023-10-16T08:16:48.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58986/0 (socket says 172.21.15.32:58986) 2023-10-16T08:16:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 -- 172.21.15.32:0/1313070428 learned_addr learned my addr 172.21.15.32:0/1313070428 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965bfff640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c0d1400 0x7f965c1663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f9660e08640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965c15cb90 0x7f965c0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 -- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c0d1400 msgr2=0x7f965c1663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c0d1400 0x7f965c1663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 -- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965c15cb90 msgr2=0x7f965c0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:48.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965c15cb90 0x7f965c0cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:48.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 -- 172.21.15.32:0/1313070428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96500094d0 con 0x7f965c0d3010 2023-10-16T08:16:48.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f965b7fe640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c156870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f965001c270 tx=0x7f965001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500b4800 con 0x7f965c0d3010 2023-10-16T08:16:48.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96500b4960 con 0x7f965c0d3010 2023-10-16T08:16:48.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500b4d10 con 0x7f965c0d3010 2023-10-16T08:16:48.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.890+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c166dd0 con 0x7f965c0d3010 2023-10-16T08:16:48.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c0cb4f0 con 0x7f965c0d3010 2023-10-16T08:16:48.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c0cb050 con 0x7f965c0d3010 2023-10-16T08:16:48.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96500b3380 con 0x7f965c0d3010 2023-10-16T08:16:48.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f9658ff9640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96480664d0 0x7f9648068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:48.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f965011e810 con 0x7f965c0d3010 2023-10-16T08:16:48.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f965bfff640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96480664d0 0x7f9648068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:48.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.894+0000 7f965bfff640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96480664d0 0x7f9648068990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f965c1fabd0 tx=0x7f964c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:48.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:48.898+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96500eba50 con 0x7f965c0d3010 2023-10-16T08:16:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.058+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f965c074010 con 0x7f965c0d3010 2023-10-16T08:16:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f9658ff9640 1 -- 172.21.15.32:0/1313070428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f96500ef070 con 0x7f965c0d3010 2023-10-16T08:16:49.067 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96480664d0 msgr2=0x7f9648068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96480664d0 0x7f9648068990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f965c1fabd0 tx=0x7f964c008040 comp rx=0 tx=0).stop 2023-10-16T08:16:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 msgr2=0x7f965c156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c156870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f965001c270 tx=0x7f965001ac80 comp rx=0 tx=0).stop 2023-10-16T08:16:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 shutdown_connections 2023-10-16T08:16:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f965c15cb90 0x7f965c0cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0d3010 0x7f965c156870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96480664d0 0x7f9648068990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 --2- 172.21.15.32:0/1313070428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f965c0d1400 0x7f965c1663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 >> 172.21.15.32:0/1313070428 conn(0x7f965c09bc50 msgr2=0x7f965c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 shutdown_connections 2023-10-16T08:16:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.062+0000 7f966286b640 1 -- 172.21.15.32:0/1313070428 wait complete. 2023-10-16T08:16:49.089 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-16T08:16:49.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 Processor -- start 2023-10-16T08:16:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 -- start start 2023-10-16T08:16:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fdd040cf090 0x7fdd040cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd040c6ad0 0x7fdd040c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd040c65f0 con 0x7fdd0415ca70 2023-10-16T08:16:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd040cfa60 con 0x7fdd040c6ad0 2023-10-16T08:16:49.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.226+0000 7fdd0b09c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd040ce820 con 0x7fdd040cf090 2023-10-16T08:16:49.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd09639640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd040c6ad0 0x7fdd040c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd08e38640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd0415ca70 0x7fdd040d3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44408/0 (socket says 172.21.15.32:44408) 2023-10-16T08:16:49.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd08e38640 1 -- 172.21.15.32:0/3135000306 learned_addr learned my addr 172.21.15.32:0/3135000306 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:49.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd03fff640 1 --2- 172.21.15.32:0/3135000306 >> v2:172.21.15.186:3300/0 conn(0x7fdd040cf090 0x7fdd040cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd09639640 1 -- 172.21.15.32:0/3135000306 >> v2:172.21.15.186:3300/0 conn(0x7fdd040cf090 msgr2=0x7fdd040cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd09639640 1 --2- 172.21.15.32:0/3135000306 >> v2:172.21.15.186:3300/0 conn(0x7fdd040cf090 0x7fdd040cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd09639640 1 -- 172.21.15.32:0/3135000306 >> v1:172.21.15.32:6789/0 conn(0x7fdd0415ca70 legacy=0x7fdd040d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd09639640 1 -- 172.21.15.32:0/3135000306 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040cdf80 con 0x7fdd040c6ad0 2023-10-16T08:16:49.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd037fe640 1 -- 172.21.15.32:0/3135000306 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329405455 0 0) 0x7fdd040c65f0 con 0x7fdd0415ca70 2023-10-16T08:16:49.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd09639640 1 --2- 172.21.15.32:0/3135000306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040c6ad0 0x7fdd040c6f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b54f0 tx=0x7fdcfc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c541239a5a0f7356 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd037fe640 1 -- 172.21.15.32:0/3135000306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0a59a0 con 0x7fdd040c6ad0 2023-10-16T08:16:49.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd037fe640 1 -- 172.21.15.32:0/3135000306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc0a5b00 con 0x7fdd040c6ad0 2023-10-16T08:16:49.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.230+0000 7fdd037fe640 1 -- 172.21.15.32:0/3135000306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0a5e90 con 0x7fdd040c6ad0 2023-10-16T08:16:49.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3135000306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040c6ad0 msgr2=0x7fdd040c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3135000306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040c6ad0 0x7fdd040c6f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b54f0 tx=0x7fdcfc01c8e0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3135000306 shutdown_connections 2023-10-16T08:16:49.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3135000306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040c6ad0 0x7fdd040c6f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3135000306 >> v2:172.21.15.186:3300/0 conn(0x7fdd040cf090 0x7fdd040cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3135000306 >> 172.21.15.32:0/3135000306 conn(0x7fdd0409bc50 msgr2=0x7fdd040d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3135000306 shutdown_connections 2023-10-16T08:16:49.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3135000306 wait complete. 2023-10-16T08:16:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 Processor -- start 2023-10-16T08:16:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- start start 2023-10-16T08:16:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd040c6ad0 0x7fdd041f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 0x7fdd0415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd0415ca70 0x7fdd0415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd041603b0 con 0x7fdd040cf090 2023-10-16T08:16:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0415e100 con 0x7fdd040c6ad0 2023-10-16T08:16:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd0b09c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd041600d0 con 0x7fdd0415ca70 2023-10-16T08:16:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 0x7fdd0415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 0x7fdd0415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59002/0 (socket says 172.21.15.32:59002) 2023-10-16T08:16:49.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 -- 172.21.15.32:0/3983976955 learned_addr learned my addr 172.21.15.32:0/3983976955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd08e38640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd040c6ad0 0x7fdd041f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd09639640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd0415ca70 0x7fdd0415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 -- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd0415ca70 msgr2=0x7fdd0415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd0415ca70 0x7fdd0415f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 -- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd040c6ad0 msgr2=0x7fdd041f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd040c6ad0 0x7fdd041f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.234+0000 7fdd03fff640 1 -- 172.21.15.32:0/3983976955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc009a70 con 0x7fdd040cf090 2023-10-16T08:16:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd03fff640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 0x7fdd0415dbc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000d800 tx=0x7fdcf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000b6c0 con 0x7fdd040cf090 2023-10-16T08:16:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000b820 con 0x7fdd040cf090 2023-10-16T08:16:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd040d38a0 con 0x7fdd040cf090 2023-10-16T08:16:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000bb90 con 0x7fdd040cf090 2023-10-16T08:16:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd041f8720 con 0x7fdd040cf090 2023-10-16T08:16:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd04074010 con 0x7fdd040cf090 2023-10-16T08:16:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdcf000bcf0 con 0x7fdd040cf090 2023-10-16T08:16:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd017fa640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdce80664d0 0x7fdce8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdcf0084670 con 0x7fdd040cf090 2023-10-16T08:16:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.238+0000 7fdd08e38640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdce80664d0 0x7fdce8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.242+0000 7fdd08e38640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdce80664d0 0x7fdce8068990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80016e0 tx=0x7fdcf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.242+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdcf00517f0 con 0x7fdd040cf090 2023-10-16T08:16:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.402+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd040c70c0 con 0x7fdd040cf090 2023-10-16T08:16:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd017fa640 1 -- 172.21.15.32:0/3983976955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fdcf0054e10 con 0x7fdd040cf090 2023-10-16T08:16:49.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:49.411 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdce80664d0 msgr2=0x7fdce8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdce80664d0 0x7fdce8068990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80016e0 tx=0x7fdcf8009040 comp rx=0 tx=0).stop 2023-10-16T08:16:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 msgr2=0x7fdd0415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 0x7fdd0415dbc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000d800 tx=0x7fdcf000dcc0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 shutdown_connections 2023-10-16T08:16:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdce80664d0 0x7fdce8068990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdd0415ca70 0x7fdd0415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd040cf090 0x7fdd0415dbc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 --2- 172.21.15.32:0/3983976955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd040c6ad0 0x7fdd041f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 >> 172.21.15.32:0/3983976955 conn(0x7fdd0409bc50 msgr2=0x7fdd040ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 shutdown_connections 2023-10-16T08:16:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.406+0000 7fdd0b09c640 1 -- 172.21.15.32:0/3983976955 wait complete. 2023-10-16T08:16:49.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-16T08:16:49.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-16T08:16:49.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-16T08:16:49.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-16T08:16:49.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-16T08:16:49.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-16T08:16:49.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-16T08:16:49.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-16T08:16:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.670+0000 7f602b356640 1 Processor -- start 2023-10-16T08:16:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.670+0000 7f602b356640 1 -- start start 2023-10-16T08:16:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f602b356640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f602415c910 0x7f60240d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f602b356640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60240ce730 con 0x7f60240cefe0 2023-10-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f602b356640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60240d0dd0 con 0x7f60240d3170 2023-10-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f602b356640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60240ce540 con 0x7f602415c910 2023-10-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029b53640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f60240d3170 0x7f60240cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37348/0 (socket says 172.21.15.32:37348) 2023-10-16T08:16:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029b53640 1 -- 172.21.15.32:0/4104735599 learned_addr learned my addr 172.21.15.32:0/4104735599 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747209129 0 0) 0x7f60240d0dd0 con 0x7f60240d3170 2023-10-16T08:16:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6010003530 con 0x7f60240d3170 2023-10-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1354350788 0 0) 0x7f6010003530 con 0x7f60240d3170 2023-10-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f602415c910 msgr2=0x7f60240d1260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f602415c910 0x7f60240d1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 >> v1:172.21.15.32:6789/0 conn(0x7f60240cefe0 legacy=0x7f602406b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240d04a0 con 0x7f60240d3170 2023-10-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f6014001680 con 0x7f60240d3170 2023-10-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 >> v1:172.21.15.186:6789/0 conn(0x7f60240d3170 legacy=0x7f60240cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6010003a60 0x7f6010003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60100047d0 0x7f6010004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6010005390 0x7f601000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601000c240 con 0x7f6010005390 2023-10-16T08:16:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601000c3b0 con 0x7f6010003a60 2023-10-16T08:16:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601000c570 con 0x7f60100047d0 2023-10-16T08:16:49.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029b53640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60100047d0 0x7f6010004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f602a354640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6010003a60 0x7f6010003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f602ab55640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6010005390 0x7f601000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029b53640 1 -- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6010003a60 msgr2=0x7f6010003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029b53640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6010003a60 0x7f6010003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.674+0000 7f6029b53640 1 -- 172.21.15.32:0/4104735599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6010005390 msgr2=0x7f601000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.678+0000 7f6029b53640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6010005390 0x7f601000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.678+0000 7f6029b53640 1 -- 172.21.15.32:0/4104735599 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240d04a0 con 0x7f60100047d0 2023-10-16T08:16:49.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.682+0000 7f6029b53640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60100047d0 0x7f6010004c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f601400c260 tx=0x7f6014027b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba05d8e943b2fc00 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.682+0000 7f55e9a68640 1 Processor -- start 2023-10-16T08:16:49.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.682+0000 7f55e9a68640 1 -- start start 2023-10-16T08:16:49.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014005ae0 con 0x7f60100047d0 2023-10-16T08:16:49.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6014005c40 con 0x7f60100047d0 2023-10-16T08:16:49.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f6029352640 1 -- 172.21.15.32:0/4104735599 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014007070 con 0x7f60100047d0 2023-10-16T08:16:49.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 Processor -- start 2023-10-16T08:16:49.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 -- start start 2023-10-16T08:16:49.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e9a68640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f55e40d15d0 0x7f55e40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e9a68640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e40ce770 con 0x7f55e415cb90 2023-10-16T08:16:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e9a68640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e40d0600 con 0x7f55e40d3010 2023-10-16T08:16:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e9a68640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e40c6450 con 0x7f55e40d15d0 2023-10-16T08:16:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55e415cb90 0x7f55e40d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44420/0 (socket says 172.21.15.32:44420) 2023-10-16T08:16:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e37fe640 1 -- 172.21.15.32:0/1958406636 learned_addr learned my addr 172.21.15.32:0/1958406636 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e2ffd640 1 --2- 172.21.15.32:0/1958406636 >> v2:172.21.15.186:3300/0 conn(0x7f55e40d15d0 0x7f55e40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602b356640 1 -- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60100047d0 msgr2=0x7f6010004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602b356640 1 --2- 172.21.15.32:0/4104735599 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60100047d0 0x7f6010004c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f601400c260 tx=0x7f6014027b60 comp rx=0 tx=0).stop 2023-10-16T08:16:49.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcc40e2d40 0x7fdcc40546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e2ffd640 1 -- 172.21.15.32:0/1958406636 >> v1:172.21.15.186:6789/0 conn(0x7f55e40d3010 legacy=0x7f55e40c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd0eaa640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcc40e2d40 0x7fdcc40546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd0eaa640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcc40e2d40 0x7fdcc40546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59020/0 (socket says 172.21.15.32:59020) 2023-10-16T08:16:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd0eaa640 1 -- 172.21.15.32:0/2478826364 learned_addr learned my addr 172.21.15.32:0/2478826364 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc4006f90 0x7fdcc40389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602ab55640 1 -- 172.21.15.32:0/4104735599 reap_dead start 2023-10-16T08:16:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602b356640 1 -- 172.21.15.32:0/4104735599 shutdown_connections 2023-10-16T08:16:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602b356640 1 -- 172.21.15.32:0/4104735599 >> 172.21.15.32:0/4104735599 conn(0x7f602409bb50 msgr2=0x7f60240d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602b356640 1 -- 172.21.15.32:0/4104735599 shutdown_connections 2023-10-16T08:16:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e2ffd640 1 -- 172.21.15.32:0/1958406636 >> v1:172.21.15.32:6789/0 conn(0x7f55e415cb90 legacy=0x7f55e40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f55e2ffd640 1 -- 172.21.15.32:0/1958406636 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e40d08a0 con 0x7f55e40d15d0 2023-10-16T08:16:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc40e3120 con 0x7fdcc4033d50 2023-10-16T08:16:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc4007570 con 0x7fdcc40e2d40 2023-10-16T08:16:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd16ab640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc4006f90 0x7fdcc40389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc404ffa0 con 0x7fdcc4006f90 2023-10-16T08:16:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7fdcd0eaa640 1 -- 172.21.15.32:0/2478826364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc4006f90 msgr2=0x7fdcc40389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7f900cfd9640 1 Processor -- start 2023-10-16T08:16:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7fdcd0eaa640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc4006f90 0x7fdcc40389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7fdcd0eaa640 1 -- 172.21.15.32:0/2478826364 >> v1:172.21.15.32:6789/0 conn(0x7fdcc4033d50 legacy=0x7fdcc4006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.686+0000 7f602b356640 1 -- 172.21.15.32:0/4104735599 wait complete. 2023-10-16T08:16:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7f602b356640 1 Processor -- start 2023-10-16T08:16:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7f602b356640 1 -- start start 2023-10-16T08:16:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7fdcd0eaa640 1 -- 172.21.15.32:0/2478826364 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4050280 con 0x7fdcc40e2d40 2023-10-16T08:16:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.690+0000 7f900cfd9640 1 -- start start 2023-10-16T08:16:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f55e2ffd640 1 --2- 172.21.15.32:0/1958406636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d15d0 0x7f55e40cfb10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d80097c0 tx=0x7f55d801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810faaa752dd1933 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60240cb1c0 0x7f60241ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240cc630 0x7f602415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602415d4e0 0x7f60240c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602415d970 con 0x7f60240cc630 2023-10-16T08:16:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602415e670 con 0x7f602415d4e0 2023-10-16T08:16:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60241f9540 con 0x7f60240cb1c0 2023-10-16T08:16:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900cfd9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f900815cf00 0x7f90080cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900cfd9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90080cdee0 0x7f90080d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900cfd9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90080ce340 con 0x7f90080d1240 2023-10-16T08:16:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900cfd9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900815d2e0 con 0x7f900815cf00 2023-10-16T08:16:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900cfd9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90080d1690 con 0x7f90080cdee0 2023-10-16T08:16:49.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60240cb1c0 0x7f60241ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60240cb1c0 0x7f60241ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36374/0 (socket says 172.21.15.32:36374) 2023-10-16T08:16:49.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 -- 172.21.15.32:0/3689661576 learned_addr learned my addr 172.21.15.32:0/3689661576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602ab55640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602415d4e0 0x7f60240c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90080cdee0 0x7f90080d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90080cdee0 0x7f90080d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41850/0 (socket says 172.21.15.32:41850) 2023-10-16T08:16:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 -- 172.21.15.32:0/331316990 learned_addr learned my addr 172.21.15.32:0/331316990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f6029b53640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240cc630 0x7f602415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 -- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602415d4e0 msgr2=0x7f60240c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f602415d4e0 0x7f60240c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 -- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240cc630 msgr2=0x7f602415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240cc630 0x7f602415cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 -- 172.21.15.32:0/3689661576 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60241fffc0 con 0x7f60240cb1c0 2023-10-16T08:16:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900659c640 1 --2- 172.21.15.32:0/331316990 >> v2:172.21.15.32:3300/0 conn(0x7f900815cf00 0x7f90080cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602a354640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60240cb1c0 0x7f60241ffa80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fd0 tx=0x7f602000ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 -- 172.21.15.32:0/331316990 >> v2:172.21.15.32:3300/0 conn(0x7f900815cf00 msgr2=0x7f90080cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 --2- 172.21.15.32:0/331316990 >> v2:172.21.15.32:3300/0 conn(0x7f900815cf00 0x7f90080cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 -- 172.21.15.32:0/331316990 >> v1:172.21.15.186:6789/0 conn(0x7f90080d1240 legacy=0x7f900806ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f9005d9b640 1 -- 172.21.15.32:0/331316990 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080d1930 con 0x7f90080cdee0 2023-10-16T08:16:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f900659c640 1 --2- 172.21.15.32:0/331316990 >> v2:172.21.15.32:3300/0 conn(0x7f900815cf00 0x7f90080cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020013680 con 0x7f60240cb1c0 2023-10-16T08:16:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7fdcd0eaa640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc40e2d40 0x7fdcc40546b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009870 tx=0x7fdcb801c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c031817ce395791e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7fdfb2bab640 1 Processor -- start 2023-10-16T08:16:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7fdfb2bab640 1 -- start start 2023-10-16T08:16:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7fdccb7fe640 1 -- 172.21.15.32:0/2478826364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80aa610 con 0x7fdcc40e2d40 2023-10-16T08:16:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60200137e0 con 0x7f60240cb1c0 2023-10-16T08:16:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020013b90 con 0x7f60240cb1c0 2023-10-16T08:16:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602415e8c0 con 0x7f60240cb1c0 2023-10-16T08:16:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602415ee80 con 0x7f60240cb1c0 2023-10-16T08:16:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7fbbe51d9640 1 Processor -- start 2023-10-16T08:16:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7fbbe51d9640 1 -- start start 2023-10-16T08:16:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdccb7fe640 1 -- 172.21.15.32:0/2478826364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb80a6040 con 0x7fdcc40e2d40 2023-10-16T08:16:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdccb7fe640 1 -- 172.21.15.32:0/2478826364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80aa990 con 0x7fdcc40e2d40 2023-10-16T08:16:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60240d0a60 con 0x7f60240cb1c0 2023-10-16T08:16:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc40e2d40 msgr2=0x7fdcc40546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdcd1eac640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc40e2d40 0x7fdcc40546b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009870 tx=0x7fdcb801c550 comp rx=0 tx=0).stop 2023-10-16T08:16:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f55e27fc640 1 -- 172.21.15.32:0/1958406636 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80ac620 con 0x7f55e40d15d0 2023-10-16T08:16:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f55e27fc640 1 -- 172.21.15.32:0/1958406636 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d80ac780 con 0x7f55e40d15d0 2023-10-16T08:16:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f55e27fc640 1 -- 172.21.15.32:0/1958406636 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80acb10 con 0x7f55e40d15d0 2023-10-16T08:16:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f55e9a68640 1 -- 172.21.15.32:0/1958406636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d15d0 msgr2=0x7f55e40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.694+0000 7f55e9a68640 1 --2- 172.21.15.32:0/1958406636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d15d0 0x7f55e40cfb10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d80097c0 tx=0x7f55d801c3c0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f55e9a68640 1 -- 172.21.15.32:0/1958406636 shutdown_connections 2023-10-16T08:16:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f55e9a68640 1 --2- 172.21.15.32:0/1958406636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d15d0 0x7f55e40cfb10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f55e9a68640 1 -- 172.21.15.32:0/1958406636 >> 172.21.15.32:0/1958406636 conn(0x7f55e409bc50 msgr2=0x7f55e40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbe51d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbbe015aa30 0x7fbbe00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbe51d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbe00cfb80 0x7fbbe00cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbe51d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe015ae10 con 0x7fbbe015e4b0 2023-10-16T08:16:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbe51d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe00d04e0 con 0x7fbbe00cfb80 2023-10-16T08:16:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbe51d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe00cb9e0 con 0x7fbbe015aa30 2023-10-16T08:16:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfb2bab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 0x7fdfac0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfb2bab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdfac0d1240 0x7fdfac0b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfb2bab640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0d16d0 con 0x7fdfac15cf00 2023-10-16T08:16:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfb2bab640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac15d2e0 con 0x7fdfac0d1240 2023-10-16T08:16:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfb2bab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0d1930 con 0x7fdfac0ce0b0 2023-10-16T08:16:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 0x7fdfac0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 0x7fdfac0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36390/0 (socket says 172.21.15.32:36390) 2023-10-16T08:16:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 -- 172.21.15.32:0/526948661 learned_addr learned my addr 172.21.15.32:0/526948661 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f55e9a68640 1 -- 172.21.15.32:0/1958406636 shutdown_connections 2023-10-16T08:16:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f55e9a68640 1 -- 172.21.15.32:0/1958406636 wait complete. 2023-10-16T08:16:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbde59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbbe015e4b0 0x7fbbe00b2750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44442/0 (socket says 172.21.15.32:44442) 2023-10-16T08:16:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbde59c640 1 -- 172.21.15.32:0/718705989 learned_addr learned my addr 172.21.15.32:0/718705989 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbded9d640 1 --2- 172.21.15.32:0/718705989 >> v2:172.21.15.186:3300/0 conn(0x7fbbe015aa30 0x7fbbe00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbdf59e640 1 --2- 172.21.15.32:0/718705989 >> v2:172.21.15.32:3300/0 conn(0x7fbbe00cfb80 0x7fbbe00cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6020013cf0 con 0x7f60240cb1c0 2023-10-16T08:16:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 shutdown_connections 2023-10-16T08:16:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdcd1eac640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc4006f90 0x7fdcc40389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdcd1eac640 1 --2- 172.21.15.32:0/2478826364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc40e2d40 0x7fdcc40546b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 >> 172.21.15.32:0/2478826364 conn(0x7fdcc4054c20 msgr2=0x7fdcc4055030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7f9005d9b640 1 --2- 172.21.15.32:0/331316990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f90080d0d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f90080b5490 tx=0x7f8ff801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23811102cf03d9f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbdf59e640 1 -- 172.21.15.32:0/718705989 >> v2:172.21.15.186:3300/0 conn(0x7fbbe015aa30 msgr2=0x7fbbe00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbdf59e640 1 --2- 172.21.15.32:0/718705989 >> v2:172.21.15.186:3300/0 conn(0x7fbbe015aa30 0x7fbbe00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbdf59e640 1 -- 172.21.15.32:0/718705989 >> v1:172.21.15.32:6789/0 conn(0x7fbbe015e4b0 legacy=0x7fbbe00b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fbbdf59e640 1 -- 172.21.15.32:0/718705989 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00b3ad0 con 0x7fbbe00cfb80 2023-10-16T08:16:49.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 -- 172.21.15.32:0/526948661 >> v2:172.21.15.32:3300/0 conn(0x7fdfac0d1240 msgr2=0x7fdfac0b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 --2- 172.21.15.32:0/526948661 >> v2:172.21.15.32:3300/0 conn(0x7fdfac0d1240 0x7fdfac0b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 -- 172.21.15.32:0/526948661 >> v1:172.21.15.32:6789/0 conn(0x7fdfac15cf00 legacy=0x7fdfac0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.698+0000 7fdfa3fff640 1 -- 172.21.15.32:0/526948661 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0d1c10 con 0x7fdfac0ce0b0 2023-10-16T08:16:49.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900559a640 1 -- 172.21.15.32:0/331316990 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80a7070 con 0x7f90080cdee0 2023-10-16T08:16:49.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900559a640 1 -- 172.21.15.32:0/331316990 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff80ab430 con 0x7f90080cdee0 2023-10-16T08:16:49.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900559a640 1 -- 172.21.15.32:0/331316990 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80ab7c0 con 0x7f90080cdee0 2023-10-16T08:16:49.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 shutdown_connections 2023-10-16T08:16:49.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdcd1eac640 1 -- 172.21.15.32:0/2478826364 wait complete. 2023-10-16T08:16:49.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f601affd640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f60080664d0 0x7f6008068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6020084dd0 con 0x7f60240cb1c0 2023-10-16T08:16:49.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f601affd640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7f600806fa90 0x7f6008071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 --> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60080725a0 con 0x7f600806fa90 2023-10-16T08:16:49.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6020085140 con 0x7f60240cb1c0 2023-10-16T08:16:49.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fef5d5d9640 1 Processor -- start 2023-10-16T08:16:49.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fef5d5d9640 1 -- start start 2023-10-16T08:16:49.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 -- 172.21.15.32:0/331316990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 msgr2=0x7f90080d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 --2- 172.21.15.32:0/331316990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f90080d0d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f90080b5490 tx=0x7f8ff801c430 comp rx=0 tx=0).stop 2023-10-16T08:16:49.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f6029b53640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f60080664d0 0x7f6008068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f602ab55640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7f600806fa90 0x7f6008071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdfa3fff640 1 --2- 172.21.15.32:0/526948661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 0x7fdfac0d0d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009870 tx=0x7fdf9c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10719cb1e406b3f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdfa37fe640 1 -- 172.21.15.32:0/526948661 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0a7070 con 0x7fdfac0ce0b0 2023-10-16T08:16:49.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f55e9a68640 1 Processor -- start 2023-10-16T08:16:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 -- 172.21.15.32:0/331316990 shutdown_connections 2023-10-16T08:16:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 --2- 172.21.15.32:0/331316990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f90080d0d00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 --2- 172.21.15.32:0/331316990 >> v2:172.21.15.32:3300/0 conn(0x7f900815cf00 0x7f90080cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdfa37fe640 1 -- 172.21.15.32:0/526948661 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0ab430 con 0x7fdfac0ce0b0 2023-10-16T08:16:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 -- 172.21.15.32:0/331316990 >> 172.21.15.32:0/331316990 conn(0x7f90080b4470 msgr2=0x7f90080b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 -- 172.21.15.32:0/331316990 shutdown_connections 2023-10-16T08:16:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdfa37fe640 1 -- 172.21.15.32:0/526948661 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0ab7c0 con 0x7fdfac0ce0b0 2023-10-16T08:16:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7fdcd1eac640 1 Processor -- start 2023-10-16T08:16:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f6029b53640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f60080664d0 0x7f6008068990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60140b7000 tx=0x7f60140b3070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f602ab55640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7f600806fa90 0x7f6008071ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef5d5d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef58164520 0x7fef580ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdfb2bab640 1 -- 172.21.15.32:0/526948661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 msgr2=0x7fdfac0d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 -- start start 2023-10-16T08:16:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 -- start start 2023-10-16T08:16:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.702+0000 7f900cfd9640 1 -- 172.21.15.32:0/331316990 wait complete. 2023-10-16T08:16:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f3cbf654640 1 Processor -- start 2023-10-16T08:16:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdfb2bab640 1 --2- 172.21.15.32:0/526948661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 0x7fdfac0d0d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009870 tx=0x7fdf9c01c470 comp rx=0 tx=0).stop 2023-10-16T08:16:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fbbdf59e640 1 --2- 172.21.15.32:0/718705989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00cfb80 0x7fbbe00cffa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0072570 tx=0x7fbbd401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66642c9ddbcb2f4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f3cbf654640 1 -- start start 2023-10-16T08:16:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef5d5d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fef580b07f0 0x7fef580cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef5d5d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef580c96f0 con 0x7fef580ad960 2023-10-16T08:16:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef5d5d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef58164900 con 0x7fef58164520 2023-10-16T08:16:49.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef5d5d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580cb950 con 0x7fef580b07f0 2023-10-16T08:16:49.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== osd.0 v2:172.21.15.32:6808/1724855588 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f60080725a0 con 0x7f600806fa90 2023-10-16T08:16:49.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef577fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fef580b07f0 0x7fef580cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef567fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef580ad960 0x7fef580b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44452/0 (socket says 172.21.15.32:44452) 2023-10-16T08:16:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef567fc640 1 -- 172.21.15.32:0/2075314626 learned_addr learned my addr 172.21.15.32:0/2075314626 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fbbddd9b640 1 -- 172.21.15.32:0/718705989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40ac400 con 0x7fbbe00cfb80 2023-10-16T08:16:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fbbddd9b640 1 -- 172.21.15.32:0/718705989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd40a8050 con 0x7fbbe00cfb80 2023-10-16T08:16:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fbbddd9b640 1 -- 172.21.15.32:0/718705989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40ac400 con 0x7fbbe00cfb80 2023-10-16T08:16:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55e40d15d0 0x7f55e414b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdfb2bab640 1 -- 172.21.15.32:0/526948661 shutdown_connections 2023-10-16T08:16:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdfb2bab640 1 --2- 172.21.15.32:0/526948661 >> v2:172.21.15.32:3300/0 conn(0x7fdfac0d1240 0x7fdfac0b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdfb2bab640 1 --2- 172.21.15.32:0/526948661 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac0ce0b0 0x7fdfac0d0d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdfb2bab640 1 -- 172.21.15.32:0/526948661 >> 172.21.15.32:0/526948661 conn(0x7fdfac0b4470 msgr2=0x7fdfac0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d3010 0x7f55e4154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e415cb90 0x7f55e4148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e414d370 con 0x7f55e415cb90 2023-10-16T08:16:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40b7fe0 con 0x7f55e40d3010 2023-10-16T08:16:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e9a68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40b82c0 con 0x7f55e40d15d0 2023-10-16T08:16:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55e40d15d0 0x7f55e414b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef56ffd640 1 --2- 172.21.15.32:0/2075314626 >> v2:172.21.15.32:3300/0 conn(0x7fef58164520 0x7fef580ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55e40d15d0 0x7f55e414b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36398/0 (socket says 172.21.15.32:36398) 2023-10-16T08:16:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 -- 172.21.15.32:0/2388166465 learned_addr learned my addr 172.21.15.32:0/2388166465 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e2ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d3010 0x7f55e4154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef55ffb640 1 -- 172.21.15.32:0/2075314626 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042812866 0 0) 0x7fef580c96f0 con 0x7fef580ad960 2023-10-16T08:16:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef55ffb640 1 -- 172.21.15.32:0/2075314626 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef44003530 con 0x7fef580ad960 2023-10-16T08:16:49.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f3cbf654640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb815ca80 0x7f3cb80ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f900cfd9640 1 Processor -- start 2023-10-16T08:16:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f3cbf654640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3cb80b00e0 0x7f3cb80b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f900cfd9640 1 -- start start 2023-10-16T08:16:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006f90 0x7fdcc40dc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4033d50 0x7fdcc4036e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc40e2d40 0x7fdcc40373a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdccbfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4033d50 0x7fdcc4036e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdccbfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4033d50 0x7fdcc4036e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41902/0 (socket says 172.21.15.32:41902) 2023-10-16T08:16:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdccbfff640 1 -- 172.21.15.32:0/1387532208 learned_addr learned my addr 172.21.15.32:0/1387532208 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40db3b0 con 0x7fdcc4006f90 2023-10-16T08:16:49.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40db690 con 0x7fdcc4033d50 2023-10-16T08:16:49.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fdcd1eac640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40db970 con 0x7fdcc40e2d40 2023-10-16T08:16:49.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 -- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d3010 msgr2=0x7f55e4154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e40d3010 0x7f55e4154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 -- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e415cb90 msgr2=0x7f55e4148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e415cb90 0x7f55e4148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7f55e37fe640 1 -- 172.21.15.32:0/2388166465 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8009420 con 0x7f55e40d15d0 2023-10-16T08:16:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc40e2d40 0x7fdcc40373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 -- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4033d50 msgr2=0x7fdcc4036e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdcc4033d50 0x7fdcc4036e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 -- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006f90 msgr2=0x7fdcc40dc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006f90 0x7fdcc40dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80094d0 con 0x7fdcc40e2d40 2023-10-16T08:16:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55e37fe640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55e40d15d0 0x7f55e414b950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c3c0 tx=0x7f55d400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d40075a0 con 0x7f55e40d15d0 2023-10-16T08:16:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d40057b0 con 0x7f55e40d15d0 2023-10-16T08:16:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4005b20 con 0x7f55e40d15d0 2023-10-16T08:16:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e414d000 con 0x7f55e40d15d0 2023-10-16T08:16:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e4154dc0 con 0x7f55e40d15d0 2023-10-16T08:16:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900cfd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f9008158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900cfd9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90080d1240 0x7f9008152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900cfd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900815cf00 0x7f9008152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900cfd9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900815ae60 con 0x7f90080d1240 2023-10-16T08:16:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900cfd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900815b140 con 0x7f90080cdee0 2023-10-16T08:16:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900cfd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900815b420 con 0x7f900815cf00 2023-10-16T08:16:49.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 -- 172.21.15.32:0/718705989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00cfb80 msgr2=0x7fbbe00cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 --2- 172.21.15.32:0/718705989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00cfb80 0x7fbbe00cffa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0072570 tx=0x7fbbd401c3c0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 -- 172.21.15.32:0/718705989 shutdown_connections 2023-10-16T08:16:49.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 --2- 172.21.15.32:0/718705989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00cfb80 0x7fbbe00cffa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 --2- 172.21.15.32:0/718705989 >> v2:172.21.15.186:3300/0 conn(0x7fbbe015aa30 0x7fbbe00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 -- 172.21.15.32:0/718705989 >> 172.21.15.32:0/718705989 conn(0x7fbbe00cd1c0 msgr2=0x7fbbe00cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdfb2bab640 1 -- 172.21.15.32:0/526948661 shutdown_connections 2023-10-16T08:16:49.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdfb2bab640 1 -- 172.21.15.32:0/526948661 wait complete. 2023-10-16T08:16:49.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbcbef640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3cb80b00e0 0x7f3cb80b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbdbf1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3cb80caf80 0x7f3cb80cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44466/0 (socket says 172.21.15.32:44466) 2023-10-16T08:16:49.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbcbef640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3cb80b00e0 0x7f3cb80b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41904/0 (socket says 172.21.15.32:41904) 2023-10-16T08:16:49.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef577fe640 1 -- 172.21.15.32:0/2075314626 >> v2:172.21.15.32:3300/0 conn(0x7fef58164520 msgr2=0x7fef580ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef577fe640 1 --2- 172.21.15.32:0/2075314626 >> v2:172.21.15.32:3300/0 conn(0x7fef58164520 0x7fef580ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef577fe640 1 -- 172.21.15.32:0/2075314626 >> v1:172.21.15.32:6789/0 conn(0x7fef580ad960 legacy=0x7fef580b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.706+0000 7fef577fe640 1 -- 172.21.15.32:0/2075314626 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580cbed0 con 0x7fef580b07f0 2023-10-16T08:16:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55c4000fd0 con 0x7f55e40d15d0 2023-10-16T08:16:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbd3f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb815ca80 0x7f3cb80ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbd3f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cb815ca80 0x7f3cb80ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59090/0 (socket says 172.21.15.32:59090) 2023-10-16T08:16:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcd16ab640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc40e2d40 0x7fdcc40373a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4055ba0 tx=0x7fdcc000d9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00125f0 con 0x7fdcc40e2d40 2023-10-16T08:16:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc0012750 con 0x7fdcc40e2d40 2023-10-16T08:16:49.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0012b00 con 0x7fdcc40e2d40 2023-10-16T08:16:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdcd1eac640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc40dbc50 con 0x7fdcc40e2d40 2023-10-16T08:16:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdcd1eac640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc40dc210 con 0x7fdcc40e2d40 2023-10-16T08:16:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f9006d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900815cf00 0x7f9008152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f9006d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900815cf00 0x7f9008152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36416/0 (socket says 172.21.15.32:36416) 2023-10-16T08:16:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f9008158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f9008158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41910/0 (socket says 172.21.15.32:41910) 2023-10-16T08:16:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 -- 172.21.15.32:0/2624296495 learned_addr learned my addr 172.21.15.32:0/2624296495 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 -- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900815cf00 msgr2=0x7f9008152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900815cf00 0x7f9008152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 -- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90080d1240 msgr2=0x7f9008152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90080d1240 0x7f9008152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f900659c640 1 -- 172.21.15.32:0/2624296495 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80094d0 con 0x7f90080cdee0 2023-10-16T08:16:49.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f900659c640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f9008158380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f900000cf60 tx=0x7f900000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000d840 con 0x7f90080cdee0 2023-10-16T08:16:49.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900000d9a0 con 0x7f90080cdee0 2023-10-16T08:16:49.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000dd30 con 0x7f90080cdee0 2023-10-16T08:16:49.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900815b700 con 0x7f90080cdee0 2023-10-16T08:16:49.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900815aa90 con 0x7f90080cdee0 2023-10-16T08:16:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 -- 172.21.15.32:0/718705989 shutdown_connections 2023-10-16T08:16:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdfb2bab640 1 Processor -- start 2023-10-16T08:16:49.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fdfb2bab640 1 -- start start 2023-10-16T08:16:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbf654640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb80cd230 con 0x7f3cb80caf80 2023-10-16T08:16:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbf654640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb80cbf90 con 0x7f3cb815ca80 2023-10-16T08:16:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7f3cbf654640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb80b0a90 con 0x7f3cb80b00e0 2023-10-16T08:16:49.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fef577fe640 1 --2- 172.21.15.32:0/2075314626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580b07f0 0x7fef580cc870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef580c2120 tx=0x7fef4801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5623f70f4dd3b7f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55d4005c80 con 0x7f55e40d15d0 2023-10-16T08:16:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55cbfff640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55d00664a0 0x7f55d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90080b5de0 con 0x7f90080cdee0 2023-10-16T08:16:49.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.710+0000 7fbbe51d9640 1 -- 172.21.15.32:0/718705989 wait complete. 2023-10-16T08:16:49.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fef55ffb640 1 -- 172.21.15.32:0/2075314626 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480ab650 con 0x7fef580b07f0 2023-10-16T08:16:49.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f55d4083580 con 0x7f55e40d15d0 2023-10-16T08:16:49.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55cbfff640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7f55d006fa60 0x7f55d0071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdca4000fd0 con 0x7fdcc40e2d40 2023-10-16T08:16:49.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 --> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f55d0072570 con 0x7f55d006fa60 2023-10-16T08:16:49.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3023401283 learned_addr learned my addr 172.21.15.32:0/3023401283 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:49.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d40507b0 con 0x7f55e40d15d0 2023-10-16T08:16:49.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fef55ffb640 1 -- 172.21.15.32:0/2075314626 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef480ab7b0 con 0x7fef580b07f0 2023-10-16T08:16:49.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fef55ffb640 1 -- 172.21.15.32:0/2075314626 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480abb40 con 0x7fef580b07f0 2023-10-16T08:16:49.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fbbe51d9640 1 Processor -- start 2023-10-16T08:16:49.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f55e2ffd640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55d00664a0 0x7f55d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296712031 0 0) 0x7f3cb80cd230 con 0x7f3cb80caf80 2023-10-16T08:16:49.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fbbe51d9640 1 -- start start 2023-10-16T08:16:49.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ca4003530 con 0x7f3cb80caf80 2023-10-16T08:16:49.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdcc0012c60 con 0x7fdcc40e2d40 2023-10-16T08:16:49.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdfb2bab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfac0ce0b0 0x7fdfac15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdfb2bab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfac0d1240 0x7fdfac15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdfb2bab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac15cf00 0x7fdfac152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdfb2bab640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac15b4f0 con 0x7fdfac0d1240 2023-10-16T08:16:49.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdfb2bab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac159d20 con 0x7fdfac0ce0b0 2023-10-16T08:16:49.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7fdfb2bab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac159fc0 con 0x7fdfac15cf00 2023-10-16T08:16:49.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdfb0947640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfac0ce0b0 0x7fdfac15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3454399949 0 0) 0x7f3ca4003530 con 0x7f3cb80caf80 2023-10-16T08:16:49.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 >> v2:172.21.15.186:3300/0 conn(0x7f3cb80b00e0 msgr2=0x7f3cb80b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 --2- 172.21.15.32:0/3023401283 >> v2:172.21.15.186:3300/0 conn(0x7f3cb80b00e0 0x7f3cb80b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 >> v2:172.21.15.32:3300/0 conn(0x7f3cb815ca80 msgr2=0x7f3cb80ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 --2- 172.21.15.32:0/3023401283 >> v2:172.21.15.32:3300/0 conn(0x7f3cb815ca80 0x7f3cb80ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.714+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb80cbcb0 con 0x7f3cb80caf80 2023-10-16T08:16:49.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdfb0947640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfac0ce0b0 0x7fdfac15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41914/0 (socket says 172.21.15.32:41914) 2023-10-16T08:16:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdfb0947640 1 -- 172.21.15.32:0/2141298323 learned_addr learned my addr 172.21.15.32:0/2141298323 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2075314626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580b07f0 msgr2=0x7fef580cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 --2- 172.21.15.32:0/2075314626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580b07f0 0x7fef580cc870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef580c2120 tx=0x7fef4801c430 comp rx=0 tx=0).stop 2023-10-16T08:16:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdfb1148640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac15cf00 0x7fdfac152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f55e3fff640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7f55d006fa60 0x7f55d0071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f3ca8001680 con 0x7f3cb80caf80 2023-10-16T08:16:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 >> v1:172.21.15.32:6789/0 conn(0x7f3cb80caf80 legacy=0x7f3cb80cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3ca4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca40047d0 0x7f3ca4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca4005190 0x7f3ca400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80cbf90 con 0x7f3ca4005190 2023-10-16T08:16:49.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80b0a90 con 0x7f3ca40047d0 2023-10-16T08:16:49.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca400c090 con 0x7f3ca40039b0 2023-10-16T08:16:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbdbf1640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca4005190 0x7f3ca400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbe51d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbe00cfb80 0x7fbbe0150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbe51d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbe015aa30 0x7fbbe014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9000015440 con 0x7f90080cdee0 2023-10-16T08:16:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f8ff6ffd640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fe80664f0 0x7f8fe80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbe51d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe015e4b0 0x7fbbe014f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbd3f0640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3ca4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbd3f0640 1 -- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca40047d0 msgr2=0x7f3ca4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbd3f0640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ca40047d0 0x7f3ca4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbd3f0640 1 -- 172.21.15.32:0/3023401283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca4005190 msgr2=0x7f3ca400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbd3f0640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca4005190 0x7f3ca400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbd3f0640 1 -- 172.21.15.32:0/3023401283 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4003530 con 0x7f3ca40039b0 2023-10-16T08:16:49.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdcc97fa640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdcb00664a0 0x7fdcb0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdcc00845c0 con 0x7fdcc40e2d40 2023-10-16T08:16:49.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2075314626 shutdown_connections 2023-10-16T08:16:49.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 --2- 172.21.15.32:0/2075314626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580b07f0 0x7fef580cc870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 --2- 172.21.15.32:0/2075314626 >> v2:172.21.15.32:3300/0 conn(0x7fef58164520 0x7fef580ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2075314626 >> 172.21.15.32:0/2075314626 conn(0x7fef5806c190 msgr2=0x7fef5806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f55e2ffd640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55d00664a0 0x7f55d0068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55d80ae000 tx=0x7f55d80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9000084bb0 con 0x7f90080cdee0 2023-10-16T08:16:49.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f8ff6ffd640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7f8fe806fb30 0x7f8fe8071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 --> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fe8072640 con 0x7f8fe806fb30 2023-10-16T08:16:49.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90000892a0 con 0x7f90080cdee0 2023-10-16T08:16:49.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2075314626 shutdown_connections 2023-10-16T08:16:49.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbded9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbe00cfb80 0x7fbbe0150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbded9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbe00cfb80 0x7fbbe0150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36446/0 (socket says 172.21.15.32:36446) 2023-10-16T08:16:49.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2075314626 wait complete. 2023-10-16T08:16:49.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdcd0eaa640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdcb00664a0 0x7fdcb0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdcc97fa640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7fdcb006fae0 0x7fdcb0071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdcb00725f0 con 0x7fdcb006fae0 2023-10-16T08:16:49.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcc0086be0 con 0x7fdcc40e2d40 2023-10-16T08:16:49.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbdf59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe015e4b0 0x7fbbe014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbdf59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe015e4b0 0x7fbbe014f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59132/0 (socket says 172.21.15.32:59132) 2023-10-16T08:16:49.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f55e3fff640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7f55d006fa60 0x7f55d0071ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f9005d9b640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fe80664f0 0x7f8fe80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbdbf1640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca4005190 0x7f3ca400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3023401283 reap_dead start 2023-10-16T08:16:49.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbde59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbe015aa30 0x7fbbe014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fbbde59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbe015aa30 0x7fbbe014f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41922/0 (socket says 172.21.15.32:41922) 2023-10-16T08:16:49.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdfa3fff640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfac0d1240 0x7fdfac15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.718+0000 7fdccbfff640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7fdcb006fae0 0x7fdcb0071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f9005d9b640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fe80664f0 0x7f8fe80689b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80af000 tx=0x7f8ff8004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbe51d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00c8d60 con 0x7fbbe015e4b0 2023-10-16T08:16:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbe51d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0151200 con 0x7fbbe015aa30 2023-10-16T08:16:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbe51d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0150540 con 0x7fbbe00cfb80 2023-10-16T08:16:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbded9d640 1 -- 172.21.15.32:0/4017156709 learned_addr learned my addr 172.21.15.32:0/4017156709 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdfa3fff640 1 -- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac15cf00 msgr2=0x7fdfac152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdfa3fff640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfac15cf00 0x7fdfac152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdfa3fff640 1 -- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfac0ce0b0 msgr2=0x7fdfac15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdfa3fff640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfac0ce0b0 0x7fdfac15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdfa3fff640 1 -- 172.21.15.32:0/2141298323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c0094d0 con 0x7fdfac0d1240 2023-10-16T08:16:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdcd0eaa640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdcb00664a0 0x7fdcb0068960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80ae000 tx=0x7fdcb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbde59c640 1 -- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbe00cfb80 msgr2=0x7fbbe0150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbde59c640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbe00cfb80 0x7fbbe0150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbde59c640 1 -- 172.21.15.32:0/4017156709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe015e4b0 msgr2=0x7fbbe014f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbde59c640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe015e4b0 0x7fbbe014f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbde59c640 1 -- 172.21.15.32:0/4017156709 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4009420 con 0x7fbbe015aa30 2023-10-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbded9d640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbe00cfb80 0x7fbbe0150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:49.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdfa3fff640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfac0d1240 0x7fdfac15afb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0ac000 tx=0x7fdf9c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbde59c640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbe015aa30 0x7fbbe014f450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c340 tx=0x7fbbd000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fef5d5d9640 1 Processor -- start 2023-10-16T08:16:49.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0005cf0 con 0x7fbbe015aa30 2023-10-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f9006d9d640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7f8fe806fb30 0x7f8fe8071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0005e50 con 0x7fbbe015aa30 2023-10-16T08:16:49.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00072d0 con 0x7fbbe015aa30 2023-10-16T08:16:49.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdccbfff640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7fdcb006fae0 0x7fdcb0071f20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3cbd3f0640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3ca4003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0007360 tx=0x7f3cb0025670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480844778d6e4187 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== osd.2 v2:172.21.15.32:6816/1665192800 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f55d0072570 con 0x7f55d006fa60 2023-10-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00b4670 con 0x7f3ca40039b0 2023-10-16T08:16:49.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb00b47d0 con 0x7f3ca40039b0 2023-10-16T08:16:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f9006d9d640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7f8fe806fb30 0x7f8fe8071f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fef5d5d9640 1 -- start start 2023-10-16T08:16:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe00b08a0 con 0x7fbbe015aa30 2023-10-16T08:16:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00b0e60 con 0x7fbbe015aa30 2023-10-16T08:16:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3caffff640 1 -- 172.21.15.32:0/3023401283 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00b4b60 con 0x7f3ca40039b0 2023-10-16T08:16:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3cbf654640 1 -- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 msgr2=0x7f3ca4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3cbf654640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3ca4003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0007360 tx=0x7f3cb0025670 comp rx=0 tx=0).stop 2023-10-16T08:16:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== osd.1 v2:172.21.15.32:6800/1484869129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fdcb00725f0 con 0x7fdcb006fae0 2023-10-16T08:16:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3cbf654640 1 -- 172.21.15.32:0/3023401283 shutdown_connections 2023-10-16T08:16:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3cbf654640 1 --2- 172.21.15.32:0/3023401283 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3ca4003dd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.722+0000 7f3cbf654640 1 -- 172.21.15.32:0/3023401283 >> 172.21.15.32:0/3023401283 conn(0x7f3cb806c190 msgr2=0x7f3cb806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbe00ce770 con 0x7fbbe015aa30 2023-10-16T08:16:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7f3cbf654640 1 -- 172.21.15.32:0/3023401283 shutdown_connections 2023-10-16T08:16:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7f3cbf654640 1 -- 172.21.15.32:0/3023401283 wait complete. 2023-10-16T08:16:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef5d5d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580ad960 0x7fef580b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef5d5d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef580b07f0 0x7fef580b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580ad960 0x7fef580b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580ad960 0x7fef580b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41938/0 (socket says 172.21.15.32:41938) 2023-10-16T08:16:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef5d5d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef58164520 0x7fef580c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 -- 172.21.15.32:0/2412878777 learned_addr learned my addr 172.21.15.32:0/2412878777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58154860 con 0x7fef580b07f0 2023-10-16T08:16:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581542a0 con 0x7fef580ad960 2023-10-16T08:16:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef577fe640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef58164520 0x7fef580c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58154580 con 0x7fef58164520 2023-10-16T08:16:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7f3cbf654640 1 Processor -- start 2023-10-16T08:16:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 -- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef58164520 msgr2=0x7fef580c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef58164520 0x7fef580c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 -- 172.21.15.32:0/2412878777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef580b07f0 msgr2=0x7fef580b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef580b07f0 0x7fef580b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48009420 con 0x7fef580ad960 2023-10-16T08:16:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef56ffd640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580ad960 0x7fef580b1510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef5000efc0 tx=0x7fef5000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbbd0007430 con 0x7fbbe015aa30 2023-10-16T08:16:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7f3cbf654640 1 -- start start 2023-10-16T08:16:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.726+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50009b80 con 0x7fef580ad960 2023-10-16T08:16:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5000f040 con 0x7fef580ad960 2023-10-16T08:16:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef500146a0 con 0x7fef580ad960 2023-10-16T08:16:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5815c090 con 0x7fef580ad960 2023-10-16T08:16:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef580b1a50 con 0x7fef580ad960 2023-10-16T08:16:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== osd.3 v2:172.21.15.32:6824/2556335256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8fe8072640 con 0x7f8fe806fb30 2023-10-16T08:16:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fbbc77fe640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0b4830 con 0x7fdfac0d1240 2023-10-16T08:16:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0b49b0 con 0x7fdfac0d1240 2023-10-16T08:16:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0b4d60 con 0x7fdfac0d1240 2023-10-16T08:16:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac15a2a0 con 0x7fdfac0d1240 2023-10-16T08:16:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbbd0054320 con 0x7fbbe015aa30 2023-10-16T08:16:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fbbded9d640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fbbc77fe640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbbcc070780 0x7fbbcc072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 --> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbcc0732f0 con 0x7fbbcc070780 2023-10-16T08:16:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbf654640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3cb814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbf654640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb80c1fe0 0x7f3cb8155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fbbded9d640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8003fd0 tx=0x7fbbd8009970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbf654640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cb8156500 0x7f3cb80c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbf654640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8156980 con 0x7f3cb80c1fe0 2023-10-16T08:16:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbf654640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80c7970 con 0x7f3cb8156500 2023-10-16T08:16:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbf654640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80c7c10 con 0x7f3ca40039b0 2023-10-16T08:16:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cb8156500 0x7f3cb80c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cb8156500 0x7f3cb80c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41942/0 (socket says 172.21.15.32:41942) 2023-10-16T08:16:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3218708557 learned_addr learned my addr 172.21.15.32:0/3218708557 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbcbef640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb80c1fe0 0x7f3cb8155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 msgr2=0x7f3cb814fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T08:16:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3ca40039b0 0x7f3cb814fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3218708557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb80c1fe0 msgr2=0x7f3cb8155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb80c1fe0 0x7f3cb8155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3218708557 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb80c7ef0 con 0x7f3cb8156500 2023-10-16T08:16:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbcbef640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cb80c1fe0 0x7f3cb8155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef5816de20 con 0x7fef580ad960 2023-10-16T08:16:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7f3cbdbf1640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cb8156500 0x7f3cb80c57a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8161810 tx=0x7f3ca8004eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.730+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fef50014800 con 0x7fef580ad960 2023-10-16T08:16:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fbbdf59e640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbbcc070780 0x7fbbcc072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80077a0 con 0x7f3cb8156500 2023-10-16T08:16:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8015400 con 0x7f3cb8156500 2023-10-16T08:16:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8015790 con 0x7f3cb8156500 2023-10-16T08:16:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbd00010b0 con 0x7fbbe015aa30 2023-10-16T08:16:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb80c81d0 con 0x7f3cb8156500 2023-10-16T08:16:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb814ffd0 con 0x7f3cb8156500 2023-10-16T08:16:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fbbdf59e640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbbcc070780 0x7fbbcc072c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac15b700 con 0x7fdfac0d1240 2023-10-16T08:16:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf9c0b3380 con 0x7fdfac0d1240 2023-10-16T08:16:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3cb80ccec0 con 0x7f3cb8156500 2023-10-16T08:16:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef4f7fe640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef38066570 0x7fef38068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfac0b5de0 con 0x7fdfac0d1240 2023-10-16T08:16:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef50084ba0 con 0x7fef580ad960 2023-10-16T08:16:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef567fc640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef38066570 0x7fef38068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef4f7fe640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7fef3806fbb0 0x7fef38071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef380726c0 con 0x7fef3806fbb0 2023-10-16T08:16:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef500892a0 con 0x7fef580ad960 2023-10-16T08:16:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fef577fe640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7fef3806fbb0 0x7fef38071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.734+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== osd.4 v2:172.21.15.186:6824/2991882832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fbbcc0732f0 con 0x7fbbcc070780 2023-10-16T08:16:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fef577fe640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7fef3806fbb0 0x7fef38071ff0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3ca80158f0 con 0x7f3cb8156500 2023-10-16T08:16:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fef567fc640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef38066570 0x7fef38068a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef40000f70 tx=0x7fef40007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== osd.7 v2:172.21.15.186:6800/403837937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fef380726c0 con 0x7fef3806fbb0 2023-10-16T08:16:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfa17fa640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf980664f0 0x7fdf980689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 --> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f60240d0780 con 0x7f600806fa90 2023-10-16T08:16:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdf9c11cfd0 con 0x7fdfac0d1240 2023-10-16T08:16:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfa17fa640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7fdf9806fb30 0x7fdf98071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 --> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf98072640 con 0x7fdf9806fb30 2023-10-16T08:16:49.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf9c11d350 con 0x7fdfac0d1240 2023-10-16T08:16:49.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfb0947640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf980664f0 0x7fdf980689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cadffb640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c9c066520 0x7f3c9c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3ca8086860 con 0x7f3cb8156500 2023-10-16T08:16:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cbd3f0640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c9c066520 0x7f3c9c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cadffb640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f3c9c06fb60 0x7f3c9c071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 --> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c9c072670 con 0x7f3c9c06fb60 2023-10-16T08:16:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca8088e40 con 0x7f3cb8156500 2023-10-16T08:16:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfb1148640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7fdf9806fb30 0x7fdf98071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfb0947640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf980664f0 0x7fdf980689b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80099a0 tx=0x7fdfa80090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.738+0000 7fdfb1148640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7fdf9806fb30 0x7fdf98071f70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f3cbd3f0640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c9c066520 0x7f3c9c0689e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00b8000 tx=0x7f3cb000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f601affd640 1 -- 172.21.15.32:0/3689661576 <== osd.0 v2:172.21.15.32:6808/1724855588 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f601c003c80 con 0x7f600806fa90 2023-10-16T08:16:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f3cbcbef640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f3c9c06fb60 0x7f3c9c071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== osd.5 v2:172.21.15.186:6801/3150472125 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fdf98072640 con 0x7fdf9806fb30 2023-10-16T08:16:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7f600806fa90 msgr2=0x7f6008071ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602b356640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7f600806fa90 0x7f6008071ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f60080664d0 msgr2=0x7f6008068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602b356640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f60080664d0 0x7f6008068990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60140b7000 tx=0x7f60140b3070 comp rx=0 tx=0).stop 2023-10-16T08:16:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60240cb1c0 msgr2=0x7f60241ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602b356640 1 --2- 172.21.15.32:0/3689661576 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f60240cb1c0 0x7f60241ffa80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fd0 tx=0x7f602000ea50 comp rx=0 tx=0).stop 2023-10-16T08:16:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f602ab55640 1 -- 172.21.15.32:0/3689661576 reap_dead start 2023-10-16T08:16:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f3cbcbef640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f3c9c06fb60 0x7f3c9c071fa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.742+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== osd.6 v2:172.21.15.186:6804/1135936083 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f3c9c072670 con 0x7f3c9c06fb60 2023-10-16T08:16:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.746+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 shutdown_connections 2023-10-16T08:16:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.746+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 >> 172.21.15.32:0/3689661576 conn(0x7f602409bb50 msgr2=0x7f6024160430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.746+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 shutdown_connections 2023-10-16T08:16:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.746+0000 7f602b356640 1 -- 172.21.15.32:0/3689661576 wait complete. 2023-10-16T08:16:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 --> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdca4002cc0 con 0x7fdcb006fae0 2023-10-16T08:16:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcc97fa640 1 -- 172.21.15.32:0/1387532208 <== osd.1 v2:172.21.15.32:6800/1484869129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdca4002cc0 con 0x7fdcb006fae0 2023-10-16T08:16:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7fdcb006fae0 msgr2=0x7fdcb0071f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7fdcb006fae0 0x7fdcb0071f20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdcb00664a0 msgr2=0x7fdcb0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdcb00664a0 0x7fdcb0068960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80ae000 tx=0x7fdcb8008040 comp rx=0 tx=0).stop 2023-10-16T08:16:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc40e2d40 msgr2=0x7fdcc40373a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcb6ffd640 1 --2- 172.21.15.32:0/1387532208 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdcc40e2d40 0x7fdcc40373a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4055ba0 tx=0x7fdcc000d9c0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7fdcd16ab640 1 -- 172.21.15.32:0/1387532208 reap_dead start 2023-10-16T08:16:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.754+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 --> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55c4002cc0 con 0x7f55d006fa60 2023-10-16T08:16:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55cbfff640 1 -- 172.21.15.32:0/2388166465 <== osd.2 v2:172.21.15.32:6816/1665192800 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f55c4002cc0 con 0x7f55d006fa60 2023-10-16T08:16:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7f55d006fa60 msgr2=0x7f55d0071ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7f55d006fa60 0x7f55d0071ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 shutdown_connections 2023-10-16T08:16:49.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 >> 172.21.15.32:0/1387532208 conn(0x7fdcc4054c20 msgr2=0x7fdcc40d9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 shutdown_connections 2023-10-16T08:16:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/1387532208 wait complete. 2023-10-16T08:16:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55d00664a0 msgr2=0x7f55d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f55d00664a0 0x7f55d0068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55d80ae000 tx=0x7f55d80a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:49.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55e40d15d0 msgr2=0x7f55e414b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 --2- 172.21.15.32:0/2388166465 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55e40d15d0 0x7f55e414b950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c3c0 tx=0x7f55d400c880 comp rx=0 tx=0).stop 2023-10-16T08:16:49.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e3fff640 1 -- 172.21.15.32:0/2388166465 reap_dead start 2023-10-16T08:16:49.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 shutdown_connections 2023-10-16T08:16:49.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 --> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90080b6130 con 0x7f8fe806fb30 2023-10-16T08:16:49.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 >> 172.21.15.32:0/2388166465 conn(0x7f55e409bc50 msgr2=0x7f55e40ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 shutdown_connections 2023-10-16T08:16:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f55e9a68640 1 -- 172.21.15.32:0/2388166465 wait complete. 2023-10-16T08:16:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.758+0000 7f8ff6ffd640 1 -- 172.21.15.32:0/2624296495 <== osd.3 v2:172.21.15.32:6824/2556335256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ffc001410 con 0x7f8fe806fb30 2023-10-16T08:16:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7f8fe806fb30 msgr2=0x7f8fe8071f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7f8fe806fb30 0x7f8fe8071f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fe80664f0 msgr2=0x7f8fe80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fe80664f0 0x7f8fe80689b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80af000 tx=0x7f8ff8004740 comp rx=0 tx=0).stop 2023-10-16T08:16:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 msgr2=0x7f9008158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 --2- 172.21.15.32:0/2624296495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90080cdee0 0x7f9008158380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f900000cf60 tx=0x7f900000cf90 comp rx=0 tx=0).stop 2023-10-16T08:16:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f9006d9d640 1 -- 172.21.15.32:0/2624296495 reap_dead start 2023-10-16T08:16:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 shutdown_connections 2023-10-16T08:16:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 >> 172.21.15.32:0/2624296495 conn(0x7f90080b4470 msgr2=0x7f90080b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 shutdown_connections 2023-10-16T08:16:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7f900cfd9640 1 -- 172.21.15.32:0/2624296495 wait complete. 2023-10-16T08:16:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.762+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 --> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfac160450 con 0x7fdf9806fb30 2023-10-16T08:16:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfa17fa640 1 -- 172.21.15.32:0/2141298323 <== osd.5 v2:172.21.15.186:6801/3150472125 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdfac160450 con 0x7fdf9806fb30 2023-10-16T08:16:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7fdf9806fb30 msgr2=0x7fdf98071f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7fdf9806fb30 0x7fdf98071f70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf980664f0 msgr2=0x7fdf980689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf980664f0 0x7fdf980689b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80099a0 tx=0x7fdfa80090e0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfac0d1240 msgr2=0x7fdfac15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 --2- 172.21.15.32:0/2141298323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfac0d1240 0x7fdfac15afb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0ac000 tx=0x7fdf9c019210 comp rx=0 tx=0).stop 2023-10-16T08:16:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 --> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef58074010 con 0x7fef3806fbb0 2023-10-16T08:16:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef4f7fe640 1 -- 172.21.15.32:0/2412878777 <== osd.7 v2:172.21.15.186:6800/403837937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef58074010 con 0x7fef3806fbb0 2023-10-16T08:16:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb1148640 1 -- 172.21.15.32:0/2141298323 reap_dead start 2023-10-16T08:16:49.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 shutdown_connections 2023-10-16T08:16:49.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 >> 172.21.15.32:0/2141298323 conn(0x7fdfac0b4470 msgr2=0x7fdfac0d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7fef3806fbb0 msgr2=0x7fef38071ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7fef3806fbb0 0x7fef38071ff0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef38066570 msgr2=0x7fef38068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef38066570 0x7fef38068a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef40000f70 tx=0x7fef40007900 comp rx=0 tx=0).stop 2023-10-16T08:16:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580ad960 msgr2=0x7fef580b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fef5d5d9640 1 --2- 172.21.15.32:0/2412878777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef580ad960 0x7fef580b1510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef5000efc0 tx=0x7fef5000c490 comp rx=0 tx=0).stop 2023-10-16T08:16:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.766+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 --> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbe015eff0 con 0x7fbbcc070780 2023-10-16T08:16:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fef577fe640 1 -- 172.21.15.32:0/2412878777 reap_dead start 2023-10-16T08:16:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 shutdown_connections 2023-10-16T08:16:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 >> 172.21.15.32:0/2412878777 conn(0x7fef5806c190 msgr2=0x7fef580af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 shutdown_connections 2023-10-16T08:16:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 shutdown_connections 2023-10-16T08:16:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fef5d5d9640 1 -- 172.21.15.32:0/2412878777 wait complete. 2023-10-16T08:16:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fdfb2bab640 1 -- 172.21.15.32:0/2141298323 wait complete. 2023-10-16T08:16:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fbbc77fe640 1 -- 172.21.15.32:0/4017156709 <== osd.4 v2:172.21.15.186:6824/2991882832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbbd4004140 con 0x7fbbcc070780 2023-10-16T08:16:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbbcc070780 msgr2=0x7fbbcc072c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7fbbe51d9640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbbcc070780 0x7fbbcc072c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbcc0664f0 msgr2=0x7fbbcc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbcc0664f0 0x7fbbcc0689b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8003fd0 tx=0x7fbbd8009970 comp rx=0 tx=0).stop 2023-10-16T08:16:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbe015aa30 msgr2=0x7fbbe014f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 --2- 172.21.15.32:0/4017156709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbe015aa30 0x7fbbe014f450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c340 tx=0x7fbbd000c800 comp rx=0 tx=0).stop 2023-10-16T08:16:49.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbdf59e640 1 -- 172.21.15.32:0/4017156709 reap_dead start 2023-10-16T08:16:49.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 shutdown_connections 2023-10-16T08:16:49.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 >> 172.21.15.32:0/4017156709 conn(0x7fbbe00cd1c0 msgr2=0x7fbbe0156420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 shutdown_connections 2023-10-16T08:16:49.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7fbbe51d9640 1 -- 172.21.15.32:0/4017156709 wait complete. 2023-10-16T08:16:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.770+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 --> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3cb80ccbe0 con 0x7f3c9c06fb60 2023-10-16T08:16:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.774+0000 7f3cadffb640 1 -- 172.21.15.32:0/3218708557 <== osd.6 v2:172.21.15.186:6804/1135936083 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3cb4009670 con 0x7f3c9c06fb60 2023-10-16T08:16:49.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.778+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f3c9c06fb60 msgr2=0x7f3c9c071fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.778+0000 7f3cbf654640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f3c9c06fb60 0x7f3c9c071fa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c9c066520 msgr2=0x7f3c9c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c9c066520 0x7f3c9c0689e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00b8000 tx=0x7f3cb000b040 comp rx=0 tx=0).stop 2023-10-16T08:16:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cb8156500 msgr2=0x7f3cb80c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 --2- 172.21.15.32:0/3218708557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cb8156500 0x7f3cb80c57a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8161810 tx=0x7f3ca8004eb0 comp rx=0 tx=0).stop 2023-10-16T08:16:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbdbf1640 1 -- 172.21.15.32:0/3218708557 reap_dead start 2023-10-16T08:16:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 shutdown_connections 2023-10-16T08:16:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 >> 172.21.15.32:0/3218708557 conn(0x7f3cb806c190 msgr2=0x7f3cb80abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 shutdown_connections 2023-10-16T08:16:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:49.782+0000 7f3cbf654640 1 -- 172.21.15.32:0/3218708557 wait complete. 2023-10-16T08:16:49.875 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:49.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T08:16:49.876 INFO:teuthology.orchestra.run.smithi032.stdout:55834574858 2023-10-16T08:16:49.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T08:16:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:55834574859 2023-10-16T08:16:49.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T08:16:49.879 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:49.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T08:16:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:55834574859 2023-10-16T08:16:49.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T08:16:49.882 INFO:teuthology.orchestra.run.smithi032.stdout:55834574859 2023-10-16T08:16:49.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T08:16:49.883 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:49.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T08:16:49.884 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:49.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T08:16:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.162+0000 7fa7ea0ad640 1 Processor -- start 2023-10-16T08:16:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7ea0ad640 1 -- start start 2023-10-16T08:16:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7ea0ad640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa7dc0dff30 0x7fa7dc050160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7ea0ad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7dc038fc0 0x7fa7dc03a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7ea0ad640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0e0310 con 0x7fa7dc050870 2023-10-16T08:16:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7ea0ad640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7dc04f520 con 0x7fa7dc038fc0 2023-10-16T08:16:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7ea0ad640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7dc039540 con 0x7fa7dc0dff30 2023-10-16T08:16:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7e98ac640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fa7dc050870 0x7fa7dc0527f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37372/0 (socket says 172.21.15.32:37372) 2023-10-16T08:16:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7e98ac640 1 -- 172.21.15.32:0/999048459 learned_addr learned my addr 172.21.15.32:0/999048459 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.166+0000 7fa7e90ab640 1 --2- 172.21.15.32:0/999048459 >> v2:172.21.15.186:3300/0 conn(0x7fa7dc0dff30 0x7fa7dc050160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.170+0000 7fa7e90ab640 1 -- 172.21.15.32:0/999048459 >> v2:172.21.15.32:3300/0 conn(0x7fa7dc038fc0 msgr2=0x7fa7dc03a5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:16:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.170+0000 7fa7dbfff640 1 -- 172.21.15.32:0/999048459 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791461270 0 0) 0x7fa7dc0e0310 con 0x7fa7dc050870 2023-10-16T08:16:50.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.170+0000 7fa7e88aa640 1 --2- 172.21.15.32:0/999048459 >> v2:172.21.15.32:3300/0 conn(0x7fa7dc038fc0 0x7fa7dc03a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.170+0000 7fa7e90ab640 1 --2- 172.21.15.32:0/999048459 >> v2:172.21.15.32:3300/0 conn(0x7fa7dc038fc0 0x7fa7dc03a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.170+0000 7fa7e90ab640 1 -- 172.21.15.32:0/999048459 >> v1:172.21.15.186:6789/0 conn(0x7fa7dc050870 legacy=0x7fa7dc0527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.170+0000 7fa7e90ab640 1 -- 172.21.15.32:0/999048459 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc039820 con 0x7fa7dc0dff30 2023-10-16T08:16:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe50359e640 1 Processor -- start 2023-10-16T08:16:50.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fa7e88aa640 1 --2- 172.21.15.32:0/999048459 >> v2:172.21.15.32:3300/0 conn(0x7fa7dc038fc0 0x7fa7dc03a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe50359e640 1 -- start start 2023-10-16T08:16:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe50359e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe5041652a0 0x7fe504161c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe50359e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe504165700 con 0x7fe5040b0170 2023-10-16T08:16:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe50359e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe504162180 con 0x7fe5040afd90 2023-10-16T08:16:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe50359e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe504161820 con 0x7fe5041652a0 2023-10-16T08:16:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe502d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5040b0170 0x7fe5040cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44474/0 (socket says 172.21.15.32:44474) 2023-10-16T08:16:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.174+0000 7fe502d9d640 1 -- 172.21.15.32:0/928949364 learned_addr learned my addr 172.21.15.32:0/928949364 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236442615 0 0) 0x7fe504165700 con 0x7fe5040b0170 2023-10-16T08:16:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e4003530 con 0x7fe5040b0170 2023-10-16T08:16:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426532340 0 0) 0x7fe504162180 con 0x7fe5040afd90 2023-10-16T08:16:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe504165700 con 0x7fe5040afd90 2023-10-16T08:16:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139612822 0 0) 0x7fe4e4003530 con 0x7fe5040b0170 2023-10-16T08:16:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 >> v2:172.21.15.186:3300/0 conn(0x7fe5041652a0 msgr2=0x7fe504161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fa7e90ab640 1 --2- 172.21.15.32:0/999048459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc050160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4000b70 tx=0x7fa7e409a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed401f9264b4dd58 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe501d9b640 1 --2- 172.21.15.32:0/928949364 >> v2:172.21.15.186:3300/0 conn(0x7fe5041652a0 0x7fe504161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 --2- 172.21.15.32:0/928949364 >> v2:172.21.15.186:3300/0 conn(0x7fe5041652a0 0x7fe504161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 >> v1:172.21.15.186:6789/0 conn(0x7fe5040afd90 legacy=0x7fe5040c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504161180 con 0x7fe5040b0170 2023-10-16T08:16:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fe4ec001680 con 0x7fe5040b0170 2023-10-16T08:16:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 >> v1:172.21.15.32:6789/0 conn(0x7fe5040b0170 legacy=0x7fe5040cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e4003c20 0x7fe4e4004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e40047d0 0x7fe4e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe4e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f4001260 con 0x7fe4e40052b0 2023-10-16T08:16:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe504161820 con 0x7fe4e40047d0 2023-10-16T08:16:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e400c280 con 0x7fe4e4003c20 2023-10-16T08:16:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fa7dbfff640 1 -- 172.21.15.32:0/999048459 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4126720 con 0x7fa7dc0dff30 2023-10-16T08:16:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fa7dbfff640 1 -- 172.21.15.32:0/999048459 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e41268c0 con 0x7fa7dc0dff30 2023-10-16T08:16:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fa7dbfff640 1 -- 172.21.15.32:0/999048459 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4126720 con 0x7fa7dc0dff30 2023-10-16T08:16:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe501d9b640 1 --2- 172.21.15.32:0/928949364 >> v2:172.21.15.186:3300/0 conn(0x7fe5041652a0 0x7fe504161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe502d9d640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe4e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe50259c640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e4003c20 0x7fe4e4004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe502d9d640 1 -- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e4003c20 msgr2=0x7fe4e4004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe502d9d640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe4e4003c20 0x7fe4e4004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe502d9d640 1 -- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e40047d0 msgr2=0x7fe4e4004c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:16:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe502d9d640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4e40047d0 0x7fe4e4004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.178+0000 7fe502d9d640 1 -- 172.21.15.32:0/928949364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504161180 con 0x7fe4e40052b0 2023-10-16T08:16:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.182+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/999048459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 msgr2=0x7fa7dc050160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.182+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/999048459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc050160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4000b70 tx=0x7fa7e409a850 comp rx=0 tx=0).stop 2023-10-16T08:16:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.182+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/999048459 shutdown_connections 2023-10-16T08:16:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.182+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/999048459 >> v2:172.21.15.32:3300/0 conn(0x7fa7dc038fc0 0x7fa7dc03a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.182+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/999048459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc050160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.182+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/999048459 >> 172.21.15.32:0/999048459 conn(0x7fa7dc054c20 msgr2=0x7fa7dc055030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/999048459 shutdown_connections 2023-10-16T08:16:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe502d9d640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe4e400b990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00c340 tx=0x7fe4ec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba38b70eda3ef53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe502d9d640 1 -- 172.21.15.32:0/928949364 reap_dead start 2023-10-16T08:16:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/999048459 wait complete. 2023-10-16T08:16:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec005ae0 con 0x7fe4e40052b0 2023-10-16T08:16:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0ab070 con 0x7fe4e40052b0 2023-10-16T08:16:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50159a640 1 -- 172.21.15.32:0/928949364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec005ae0 con 0x7fe4e40052b0 2023-10-16T08:16:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 -- 172.21.15.32:0/928949364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 msgr2=0x7fe4e400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe4e400b990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00c340 tx=0x7fe4ec01f3f0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 -- 172.21.15.32:0/928949364 shutdown_connections 2023-10-16T08:16:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7f0e4b0dc640 1 Processor -- start 2023-10-16T08:16:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 --2- 172.21.15.32:0/928949364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe4e400b990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7f0e4b0dc640 1 -- start start 2023-10-16T08:16:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 -- 172.21.15.32:0/928949364 >> 172.21.15.32:0/928949364 conn(0x7fe50406a4f0 msgr2=0x7fe5040cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 -- 172.21.15.32:0/928949364 shutdown_connections 2023-10-16T08:16:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fa7ea0ad640 1 Processor -- start 2023-10-16T08:16:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fe50359e640 1 -- 172.21.15.32:0/928949364 wait complete. 2023-10-16T08:16:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.186+0000 7fa7ea0ad640 1 -- start start 2023-10-16T08:16:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4b0dc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e44159520 0x7f0e440cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4b0dc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e440cc5d0 con 0x7f0e4415d870 2023-10-16T08:16:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4b0dc640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4415e090 con 0x7f0e4406c070 2023-10-16T08:16:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4b0dc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440ccfa0 con 0x7f0e44159520 2023-10-16T08:16:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4a0da640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e44159520 0x7f0e440cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4a0da640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e44159520 0x7f0e440cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41984/0 (socket says 172.21.15.32:41984) 2023-10-16T08:16:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4a0da640 1 -- 172.21.15.32:0/148542185 learned_addr learned my addr 172.21.15.32:0/148542185 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7ea0ad640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 0x7fa7dc0d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fe50359e640 1 Processor -- start 2023-10-16T08:16:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fe50359e640 1 -- start start 2023-10-16T08:16:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7ea0ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7dc050870 0x7fa7dc034e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7ea0ad640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc0353d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7ea0ad640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc035a10 con 0x7fa7dc050870 2023-10-16T08:16:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7e90ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 0x7fa7dc0d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7e90ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 0x7fa7dc0d63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36490/0 (socket says 172.21.15.32:36490) 2023-10-16T08:16:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fa7e90ab640 1 -- 172.21.15.32:0/370810164 learned_addr learned my addr 172.21.15.32:0/370810164 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc035cf0 con 0x7fa7dc0dff30 2023-10-16T08:16:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc035fd0 con 0x7fa7dc038fc0 2023-10-16T08:16:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e490d8640 1 -- 172.21.15.32:0/148542185 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446959281 0 0) 0x7f0e440cc5d0 con 0x7f0e4415d870 2023-10-16T08:16:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e490d8640 1 -- 172.21.15.32:0/148542185 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e30005740 con 0x7f0e4415d870 2023-10-16T08:16:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4a0da640 1 -- 172.21.15.32:0/148542185 >> v1:172.21.15.186:6789/0 conn(0x7f0e4406c070 legacy=0x7f0e4415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4a0da640 1 -- 172.21.15.32:0/148542185 >> v1:172.21.15.32:6789/0 conn(0x7f0e4415d870 legacy=0x7f0e44070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7f0e4a0da640 1 -- 172.21.15.32:0/148542185 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4415ddb0 con 0x7f0e44159520 2023-10-16T08:16:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fd8517a7640 1 Processor -- start 2023-10-16T08:16:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fd8517a7640 1 -- start start 2023-10-16T08:16:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fd8517a7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd84c0c1920 0x7fd84c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.190+0000 7fd84b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd84c0b04a0 0x7fd84c0cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44488/0 (socket says 172.21.15.32:44488) 2023-10-16T08:16:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe5041609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd84b7fe640 1 -- 172.21.15.32:0/3314733501 learned_addr learned my addr 172.21.15.32:0/3314733501 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c06cd20 con 0x7fd84c0b04a0 2023-10-16T08:16:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e98ac640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc0353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c06c110 con 0x7fd84c160b50 2023-10-16T08:16:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84c06c3f0 con 0x7fd84c0c1920 2023-10-16T08:16:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50359e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 0x7fe5040b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50359e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5040b3bd0 0x7fe5040c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e90ab640 1 -- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 msgr2=0x7fa7dc0353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e90ab640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc0353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e90ab640 1 -- 172.21.15.32:0/370810164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7dc050870 msgr2=0x7fa7dc034e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e90ab640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7dc050870 0x7fa7dc034e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e90ab640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e406f0c0 con 0x7fa7dc038fc0 2023-10-16T08:16:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe501d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 0x7fe5040b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe501d9b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 0x7fe5040b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42010/0 (socket says 172.21.15.32:42010) 2023-10-16T08:16:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe502d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5040b3bd0 0x7fe5040c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe502d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5040b3bd0 0x7fe5040c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36492/0 (socket says 172.21.15.32:36492) 2023-10-16T08:16:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe502d9d640 1 -- 172.21.15.32:0/1524852108 learned_addr learned my addr 172.21.15.32:0/1524852108 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50259c640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe5041609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd850fa6640 1 --2- 172.21.15.32:0/3314733501 >> v2:172.21.15.32:3300/0 conn(0x7fd84c0c1920 0x7fd84c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183789707 0 0) 0x7fd84c06c110 con 0x7fd84c160b50 2023-10-16T08:16:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd834003530 con 0x7fd84c160b50 2023-10-16T08:16:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50359e640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5040b3270 con 0x7fe4e40052b0 2023-10-16T08:16:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50359e640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5040b23a0 con 0x7fe5040c2370 2023-10-16T08:16:50.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fe50359e640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5040b25f0 con 0x7fe5040b3bd0 2023-10-16T08:16:50.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fa7e90ab640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 0x7fa7dc0d63e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40697b0 tx=0x7fa7e4097640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119885590 0 0) 0x7fd84c06cd20 con 0x7fd84c0b04a0 2023-10-16T08:16:50.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84c06c110 con 0x7fd84c0b04a0 2023-10-16T08:16:50.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd850fa6640 1 -- 172.21.15.32:0/3314733501 >> v1:172.21.15.186:6789/0 conn(0x7fd84c160b50 legacy=0x7fd84c0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe501d9b640 1 -- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5040b3bd0 msgr2=0x7fe5040c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe501d9b640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5040b3bd0 0x7fe5040c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe501d9b640 1 -- 172.21.15.32:0/1524852108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 msgr2=0x7fe5041609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd850fa6640 1 -- 172.21.15.32:0/3314733501 >> v1:172.21.15.32:6789/0 conn(0x7fd84c0b04a0 legacy=0x7fd84c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe501d9b640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe5041609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.194+0000 7fd850fa6640 1 -- 172.21.15.32:0/3314733501 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c06c6d0 con 0x7fd84c0c1920 2023-10-16T08:16:50.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe501d9b640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5040b28d0 con 0x7fe5040c2370 2023-10-16T08:16:50.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7f245ae17640 1 Processor -- start 2023-10-16T08:16:50.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4134070 con 0x7fa7dc038fc0 2023-10-16T08:16:50.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe501d9b640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 0x7fe5040b3690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00cdf0 tx=0x7fe4fc00a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7f245ae17640 1 -- start start 2023-10-16T08:16:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0362b0 con 0x7fa7dc038fc0 2023-10-16T08:16:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0d6920 con 0x7fa7dc038fc0 2023-10-16T08:16:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e412f880 con 0x7fa7dc038fc0 2023-10-16T08:16:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4134070 con 0x7fa7dc038fc0 2023-10-16T08:16:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00d840 con 0x7fe5040c2370 2023-10-16T08:16:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe50359e640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5040b2bb0 con 0x7fe5040c2370 2023-10-16T08:16:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe50359e640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5040c4d00 con 0x7fe5040c2370 2023-10-16T08:16:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc00d9a0 con 0x7fe5040c2370 2023-10-16T08:16:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00d840 con 0x7fe5040c2370 2023-10-16T08:16:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.198+0000 7f245ae17640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24540afff0 0x7f245406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c4005260 con 0x7fa7dc038fc0 2023-10-16T08:16:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459e15640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24540afff0 0x7f245406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459e15640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24540afff0 0x7f245406ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59212/0 (socket says 172.21.15.32:59212) 2023-10-16T08:16:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa7e4138070 con 0x7fa7dc038fc0 2023-10-16T08:16:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fd850fa6640 1 --2- 172.21.15.32:0/3314733501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c0c1920 0x7fd84c0ccac0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8400097c0 tx=0x7fd84001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafa88dd4078bc1b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f245ae17640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f24540ad6c0 0x7f245406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f245ae17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24540abd80 con 0x7f24540b03d0 2023-10-16T08:16:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f245ae17640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24540ab830 con 0x7f24540afff0 2023-10-16T08:16:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f245ae17640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245406c9e0 con 0x7f24540ad6c0 2023-10-16T08:16:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459e15640 1 -- 172.21.15.32:0/3335548805 learned_addr learned my addr 172.21.15.32:0/3335548805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459614640 1 --2- 172.21.15.32:0/3335548805 >> v2:172.21.15.186:3300/0 conn(0x7f24540ad6c0 0x7f245406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459614640 1 -- 172.21.15.32:0/3335548805 >> v2:172.21.15.32:3300/0 conn(0x7f24540afff0 msgr2=0x7f245406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459614640 1 --2- 172.21.15.32:0/3335548805 >> v2:172.21.15.32:3300/0 conn(0x7f24540afff0 0x7f245406ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459614640 1 -- 172.21.15.32:0/3335548805 >> v1:172.21.15.32:6789/0 conn(0x7f24540b03d0 legacy=0x7f24540ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400ab600 con 0x7fd84c0c1920 2023-10-16T08:16:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8400a7050 con 0x7fd84c0c1920 2023-10-16T08:16:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fd84affd640 1 -- 172.21.15.32:0/3314733501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400ab980 con 0x7fd84c0c1920 2023-10-16T08:16:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fa7d9ffb640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa7c8066560 0x7fa7c8068a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f2459614640 1 -- 172.21.15.32:0/3335548805 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245406ccc0 con 0x7f24540ad6c0 2023-10-16T08:16:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7e419c4b0 con 0x7fa7dc038fc0 2023-10-16T08:16:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7fa7e88aa640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa7c8066560 0x7fa7c8068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f0e4a0da640 1 --2- 172.21.15.32:0/148542185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e44159520 0x7f0e440cca60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009870 tx=0x7f0e40020910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea4ac00c79e5ab93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f0e490d8640 1 -- 172.21.15.32:0/148542185 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400a9840 con 0x7f0e44159520 2023-10-16T08:16:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f0e490d8640 1 -- 172.21.15.32:0/148542185 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e400a99a0 con 0x7f0e44159520 2023-10-16T08:16:50.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.202+0000 7f0e490d8640 1 -- 172.21.15.32:0/148542185 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400a9d30 con 0x7f0e44159520 2023-10-16T08:16:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe4fc00db50 con 0x7fe5040c2370 2023-10-16T08:16:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/148542185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e44159520 msgr2=0x7f0e440cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/148542185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e44159520 0x7f0e440cca60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009870 tx=0x7f0e40020910 comp rx=0 tx=0).stop 2023-10-16T08:16:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50404e9c0 con 0x7fe5040c2370 2023-10-16T08:16:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fa7e88aa640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa7c8066560 0x7fa7c8068a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0482b0 tx=0x7fa7e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c0c1920 msgr2=0x7fd84c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 --2- 172.21.15.32:0/3314733501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c0c1920 0x7fd84c0ccac0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8400097c0 tx=0x7fd84001c3e0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fe4faffd640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe4e8066530 0x7fe4e80689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 shutdown_connections 2023-10-16T08:16:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 --2- 172.21.15.32:0/3314733501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c0c1920 0x7fd84c0ccac0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 >> 172.21.15.32:0/3314733501 conn(0x7fd84c0cad90 msgr2=0x7fd84c0cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/148542185 shutdown_connections 2023-10-16T08:16:50.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/148542185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e44159520 0x7f0e440cca60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe4fc0845e0 con 0x7fe5040c2370 2023-10-16T08:16:50.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/148542185 >> 172.21.15.32:0/148542185 conn(0x7f0e4406a4f0 msgr2=0x7f0e440cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fe50259c640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe4e8066530 0x7fe4e80689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 shutdown_connections 2023-10-16T08:16:50.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fd8517a7640 1 -- 172.21.15.32:0/3314733501 wait complete. 2023-10-16T08:16:50.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/148542185 shutdown_connections 2023-10-16T08:16:50.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/148542185 wait complete. 2023-10-16T08:16:50.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7f2459614640 1 --2- 172.21.15.32:0/3335548805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540ad6c0 0x7f245406c070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24540723d0 tx=0x7f244401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b09ddecdd6cd815 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.206+0000 7fe50259c640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe4e8066530 0x7fe4e80689f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5040c88e0 tx=0x7fe4f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 Processor -- start 2023-10-16T08:16:50.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa7e41695b0 con 0x7fa7dc038fc0 2023-10-16T08:16:50.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 -- start start 2023-10-16T08:16:50.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 0x7f0e44144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e44159520 0x7f0e4414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e4415d870 0x7f0e44147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4414b120 con 0x7f0e44159520 2023-10-16T08:16:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44159b20 con 0x7f0e4406c070 2023-10-16T08:16:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4b0dc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44159e00 con 0x7f0e4415d870 2023-10-16T08:16:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 0x7f0e44144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 0x7f0e44144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42032/0 (socket says 172.21.15.32:42032) 2023-10-16T08:16:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 -- 172.21.15.32:0/3751035953 learned_addr learned my addr 172.21.15.32:0/3751035953 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7fd8517a7640 1 Processor -- start 2023-10-16T08:16:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a8db640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e4415d870 0x7f0e44147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7fd8517a7640 1 -- start start 2023-10-16T08:16:50.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e498d9640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e44159520 0x7f0e4414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 -- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e4415d870 msgr2=0x7f0e44147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e4415d870 0x7f0e44147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 -- 172.21.15.32:0/3751035953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e44159520 msgr2=0x7f0e4414aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e44159520 0x7f0e4414aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e4a0da640 1 -- 172.21.15.32:0/3751035953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e400094d0 con 0x7f0e4406c070 2023-10-16T08:16:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.210+0000 7f0e498d9640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e44159520 0x7f0e4414aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f0e4a0da640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 0x7f0e44144c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b4040 tx=0x7f0e4001d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd8517a7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd84c0b04a0 0x7fd84c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd8517a7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 0x7fd84c0c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd8517a7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c160b50 0x7fd84c0c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400b1850 con 0x7f0e4406c070 2023-10-16T08:16:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e400b19b0 con 0x7f0e4406c070 2023-10-16T08:16:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400b1d60 con 0x7f0e4406c070 2023-10-16T08:16:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 0x7fd84c0c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 0x7fd84c0c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42036/0 (socket says 172.21.15.32:42036) 2023-10-16T08:16:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 -- 172.21.15.32:0/4180101896 learned_addr learned my addr 172.21.15.32:0/4180101896 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0c2240 con 0x7fd84c160b50 2023-10-16T08:16:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0c3060 con 0x7fd84c0c1920 2023-10-16T08:16:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0c2450 con 0x7fd84c0b04a0 2023-10-16T08:16:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f2458e13640 1 -- 172.21.15.32:0/3335548805 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440ab600 con 0x7f24540ad6c0 2023-10-16T08:16:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f2458e13640 1 -- 172.21.15.32:0/3335548805 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440a7050 con 0x7f24540ad6c0 2023-10-16T08:16:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f2458e13640 1 -- 172.21.15.32:0/3335548805 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440ab980 con 0x7f24540ad6c0 2023-10-16T08:16:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd850fa6640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c160b50 0x7fd84c0c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 -- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd84c0b04a0 msgr2=0x7fd84c14d110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T08:16:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd84c0b04a0 0x7fd84c14d110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 -- 172.21.15.32:0/4180101896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c160b50 msgr2=0x7fd84c0c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c160b50 0x7fd84c0c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840009420 con 0x7fd84c0c1920 2023-10-16T08:16:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4415a0e0 con 0x7f0e4406c070 2023-10-16T08:16:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e44148910 con 0x7f0e4406c070 2023-10-16T08:16:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe4fc051810 con 0x7fe5040c2370 2023-10-16T08:16:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f245ae17640 1 -- 172.21.15.32:0/3335548805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540ad6c0 msgr2=0x7f245406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f245ae17640 1 --2- 172.21.15.32:0/3335548805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540ad6c0 0x7f245406c070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24540723d0 tx=0x7f244401c460 comp rx=0 tx=0).stop 2023-10-16T08:16:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f245ae17640 1 -- 172.21.15.32:0/3335548805 shutdown_connections 2023-10-16T08:16:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f245ae17640 1 --2- 172.21.15.32:0/3335548805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540ad6c0 0x7f245406c070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd84b7fe640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 0x7fd84c0c7890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00c4d0 tx=0x7fd83c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f245ae17640 1 --2- 172.21.15.32:0/3335548805 >> v2:172.21.15.32:3300/0 conn(0x7f24540afff0 0x7f245406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7f245ae17640 1 -- 172.21.15.32:0/3335548805 >> 172.21.15.32:0/3335548805 conn(0x7f24540cad90 msgr2=0x7f24540cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4404e9c0 con 0x7f0e4406c070 2023-10-16T08:16:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.214+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c005cf0 con 0x7fd84c0c1920 2023-10-16T08:16:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c005e50 con 0x7fd84c0c1920 2023-10-16T08:16:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c007270 con 0x7fd84c0c1920 2023-10-16T08:16:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c0c2730 con 0x7fd84c0c1920 2023-10-16T08:16:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7f245ae17640 1 -- 172.21.15.32:0/3335548805 shutdown_connections 2023-10-16T08:16:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c0c2cf0 con 0x7fd84c0c1920 2023-10-16T08:16:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c005260 con 0x7fd84c0c1920 2023-10-16T08:16:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7f245ae17640 1 -- 172.21.15.32:0/3335548805 wait complete. 2023-10-16T08:16:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.218+0000 7f245ae17640 1 Processor -- start 2023-10-16T08:16:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0e400b8070 con 0x7f0e4406c070 2023-10-16T08:16:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f0e3affd640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e28066570 0x7f0e28068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e40122dc0 con 0x7f0e4406c070 2023-10-16T08:16:50.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd83c0073d0 con 0x7fd84c0c1920 2023-10-16T08:16:50.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7fd848ff9640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd838066570 0x7fd838068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd83c083770 con 0x7fd84c0c1920 2023-10-16T08:16:50.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 -- start start 2023-10-16T08:16:50.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24540ad6c0 0x7f245416b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 0x7f245416c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f0e498d9640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e28066570 0x7f0e28068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540b03d0 0x7f2454151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24540b1540 con 0x7f24540afff0 2023-10-16T08:16:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24540b0ec0 con 0x7f24540b03d0 2023-10-16T08:16:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f245ae17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245416ce80 con 0x7f24540ad6c0 2023-10-16T08:16:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f2459614640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 0x7f245416c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0e400eff30 con 0x7f0e4406c070 2023-10-16T08:16:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f2459614640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 0x7f245416c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59244/0 (socket says 172.21.15.32:59244) 2023-10-16T08:16:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f2459614640 1 -- 172.21.15.32:0/1977421307 learned_addr learned my addr 172.21.15.32:0/1977421307 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.222+0000 7f2459e15640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24540ad6c0 0x7f245416b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f245a616640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540b03d0 0x7f2454151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459e15640 1 -- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540b03d0 msgr2=0x7f2454151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459e15640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540b03d0 0x7f2454151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459e15640 1 -- 172.21.15.32:0/1977421307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 msgr2=0x7f245416c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459e15640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 0x7f245416c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459e15640 1 -- 172.21.15.32:0/1977421307 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444009420 con 0x7f24540ad6c0 2023-10-16T08:16:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459614640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 0x7f245416c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f2459e15640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24540ad6c0 0x7f245416b8e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f245000e8a0 tx=0x7f245000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f0e498d9640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e28066570 0x7f0e28068a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e440c4f30 tx=0x7f0e3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000c940 con 0x7f24540ad6c0 2023-10-16T08:16:50.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245000caa0 con 0x7f24540ad6c0 2023-10-16T08:16:50.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000ce30 con 0x7f24540ad6c0 2023-10-16T08:16:50.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f245ae17640 1 -- 172.21.15.32:0/1977421307 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245416bf00 con 0x7f24540ad6c0 2023-10-16T08:16:50.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f245ae17640 1 -- 172.21.15.32:0/1977421307 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24540b1160 con 0x7f24540ad6c0 2023-10-16T08:16:50.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7f245ae17640 1 -- 172.21.15.32:0/1977421307 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245404e9c0 con 0x7f24540ad6c0 2023-10-16T08:16:50.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7fd84bfff640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd838066570 0x7fd838068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.226+0000 7fd84bfff640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd838066570 0x7fd838068a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd844001be0 tx=0x7fd844007c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.230+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2450013470 con 0x7f24540ad6c0 2023-10-16T08:16:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.230+0000 7f244a7fc640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f24380664a0 0x7f2438068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.230+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2450084330 con 0x7f24540ad6c0 2023-10-16T08:16:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.230+0000 7f2459614640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f24380664a0 0x7f2438068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.234+0000 7f2459614640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f24380664a0 0x7f2438068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24540c9510 tx=0x7f2444008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.234+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd83c050900 con 0x7fd84c0c1920 2023-10-16T08:16:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f24500514b0 con 0x7f24540ad6c0 2023-10-16T08:16:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 Processor -- start 2023-10-16T08:16:50.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 -- start start 2023-10-16T08:16:50.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 Processor -- start 2023-10-16T08:16:50.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 -- start start 2023-10-16T08:16:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc18415eaf0 0x7fc1841655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc1840cf270 0x7fc184161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1841625e0 con 0x7fc1840ccfb0 2023-10-16T08:16:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc184162020 con 0x7fc18415eaf0 2023-10-16T08:16:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc18a4c9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc184162300 con 0x7fc1840cf270 2023-10-16T08:16:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc096e80 0x7f9acc158b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9acc0ce730 0x7f9acc0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc159060 con 0x7f9acc071f20 2023-10-16T08:16:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc097e90 con 0x7f9acc0ce730 2023-10-16T08:16:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad2510640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc097310 con 0x7f9acc096e80 2023-10-16T08:16:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7f9ad0d0d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9acc071f20 0x7f9acc0ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44496/0 (socket says 172.21.15.32:44496) 2023-10-16T08:16:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9ad0d0d640 1 -- 172.21.15.32:0/2202738790 learned_addr learned my addr 172.21.15.32:0/2202738790 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9ad1d0f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9acc0ce730 0x7f9acc0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9acbfff640 1 -- 172.21.15.32:0/2202738790 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375120913 0 0) 0x7f9acc159060 con 0x7f9acc071f20 2023-10-16T08:16:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9acbfff640 1 -- 172.21.15.32:0/2202738790 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9abc003530 con 0x7f9acc071f20 2023-10-16T08:16:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9acbfff640 1 -- 172.21.15.32:0/2202738790 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483238373 0 0) 0x7f9abc003530 con 0x7f9acc071f20 2023-10-16T08:16:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc183fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc18415eaf0 0x7fc1841655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc183fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc18415eaf0 0x7fc1841655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59252/0 (socket says 172.21.15.32:59252) 2023-10-16T08:16:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.238+0000 7fc183fff640 1 -- 172.21.15.32:0/3478366105 learned_addr learned my addr 172.21.15.32:0/3478366105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7fc183fff640 1 -- 172.21.15.32:0/3478366105 >> v2:172.21.15.186:3300/0 conn(0x7fc1840cf270 msgr2=0x7fc184161740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:16:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7fc183fff640 1 --2- 172.21.15.32:0/3478366105 >> v2:172.21.15.186:3300/0 conn(0x7fc1840cf270 0x7fc184161740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7fc183fff640 1 -- 172.21.15.32:0/3478366105 >> v1:172.21.15.186:6789/0 conn(0x7fc1840ccfb0 legacy=0x7fc1840d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7fc183fff640 1 -- 172.21.15.32:0/3478366105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc184164df0 con 0x7fc18415eaf0 2023-10-16T08:16:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9ad1d0f640 1 -- 172.21.15.32:0/2202738790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc096e80 msgr2=0x7f9acc158b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:16:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9ad1d0f640 1 --2- 172.21.15.32:0/2202738790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc096e80 0x7f9acc158b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9ad1d0f640 1 -- 172.21.15.32:0/2202738790 >> v1:172.21.15.32:6789/0 conn(0x7f9acc071f20 legacy=0x7f9acc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.242+0000 7f9ad1d0f640 1 -- 172.21.15.32:0/2202738790 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0975f0 con 0x7f9acc0ce730 2023-10-16T08:16:50.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7fc183fff640 1 --2- 172.21.15.32:0/3478366105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415eaf0 0x7fc1841655f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009e50 tx=0x7fc17c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0f9702d6cafdf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7fc182ffd640 1 -- 172.21.15.32:0/3478366105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0a57d0 con 0x7fc18415eaf0 2023-10-16T08:16:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7fc182ffd640 1 -- 172.21.15.32:0/3478366105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0a5930 con 0x7fc18415eaf0 2023-10-16T08:16:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7fc182ffd640 1 -- 172.21.15.32:0/3478366105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0a5cc0 con 0x7fc18415eaf0 2023-10-16T08:16:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7f9ad1d0f640 1 --2- 172.21.15.32:0/2202738790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0adf80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ba50 tx=0x7f9ac401ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e4e47fab19afa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdff2dc640 1 Processor -- start 2023-10-16T08:16:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdff2dc640 1 -- start start 2023-10-16T08:16:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdff2dc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdf80cb370 0x7efdf815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdff2dc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efdf80cc740 0x7efdf80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdfe2da640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7efdf8158580 0x7efdf80cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37398/0 (socket says 172.21.15.32:37398) 2023-10-16T08:16:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdfe2da640 1 -- 172.21.15.32:0/3499735440 learned_addr learned my addr 172.21.15.32:0/3499735440 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdfeadb640 1 --2- 172.21.15.32:0/3499735440 >> v2:172.21.15.186:3300/0 conn(0x7efdf80cc740 0x7efdf80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdfdad9640 1 --2- 172.21.15.32:0/3499735440 >> v2:172.21.15.32:3300/0 conn(0x7efdf80cb370 0x7efdf815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.246+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf8159de0 con 0x7efdf8158580 2023-10-16T08:16:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7f9acbfff640 1 -- 172.21.15.32:0/2202738790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8001680 con 0x7f9acc0ce730 2023-10-16T08:16:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf8159b00 con 0x7efdf80cb370 2023-10-16T08:16:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf815c560 con 0x7efdf80cc740 2023-10-16T08:16:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7f9acbfff640 1 -- 172.21.15.32:0/2202738790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac40ab070 con 0x7f9acc0ce730 2023-10-16T08:16:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdfd2d8640 1 -- 172.21.15.32:0/3499735440 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059469639 0 0) 0x7efdf8159de0 con 0x7efdf8158580 2023-10-16T08:16:50.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdfeadb640 1 -- 172.21.15.32:0/3499735440 >> v2:172.21.15.32:3300/0 conn(0x7efdf80cb370 msgr2=0x7efdf815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:50.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdfeadb640 1 --2- 172.21.15.32:0/3499735440 >> v2:172.21.15.32:3300/0 conn(0x7efdf80cb370 0x7efdf815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdfeadb640 1 -- 172.21.15.32:0/3499735440 >> v1:172.21.15.186:6789/0 conn(0x7efdf8158580 legacy=0x7efdf80cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7f9ad2510640 1 -- 172.21.15.32:0/2202738790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 msgr2=0x7f9acc0adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7f9acbfff640 1 -- 172.21.15.32:0/2202738790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8001680 con 0x7f9acc0ce730 2023-10-16T08:16:50.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7f9ad2510640 1 --2- 172.21.15.32:0/2202738790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0adf80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ba50 tx=0x7f9ac401ea30 comp rx=0 tx=0).stop 2023-10-16T08:16:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7efdfeadb640 1 -- 172.21.15.32:0/3499735440 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf815c840 con 0x7efdf80cc740 2023-10-16T08:16:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7fc18a4c9640 1 -- 172.21.15.32:0/3478366105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415eaf0 msgr2=0x7fc1841655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/3478366105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415eaf0 0x7fc1841655f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009e50 tx=0x7fc17c01c930 comp rx=0 tx=0).stop 2023-10-16T08:16:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7fc18a4c9640 1 -- 172.21.15.32:0/3478366105 shutdown_connections 2023-10-16T08:16:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/3478366105 >> v2:172.21.15.186:3300/0 conn(0x7fc1840cf270 0x7fc184161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/3478366105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415eaf0 0x7fc1841655f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 -- 172.21.15.32:0/2202738790 shutdown_connections 2023-10-16T08:16:50.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 --2- 172.21.15.32:0/2202738790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0adf80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 --2- 172.21.15.32:0/2202738790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc096e80 0x7f9acc158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 -- 172.21.15.32:0/2202738790 >> 172.21.15.32:0/2202738790 conn(0x7f9acc071400 msgr2=0x7f9acc071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.250+0000 7fc18a4c9640 1 -- 172.21.15.32:0/3478366105 >> 172.21.15.32:0/3478366105 conn(0x7fc1840b4470 msgr2=0x7fc1840b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7fc18a4c9640 1 -- 172.21.15.32:0/3478366105 shutdown_connections 2023-10-16T08:16:50.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 -- 172.21.15.32:0/2202738790 shutdown_connections 2023-10-16T08:16:50.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 -- 172.21.15.32:0/2202738790 wait complete. 2023-10-16T08:16:50.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7f9ad2510640 1 Processor -- start 2023-10-16T08:16:50.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7f9ad2510640 1 -- start start 2023-10-16T08:16:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.254+0000 7fc18a4c9640 1 -- 172.21.15.32:0/3478366105 wait complete. 2023-10-16T08:16:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 Processor -- start 2023-10-16T08:16:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7efdfeadb640 1 --2- 172.21.15.32:0/3499735440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf80ccbf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdf80ca6a0 tx=0x7efdf001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a0afbd8e6432f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 -- start start 2023-10-16T08:16:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1840ccfb0 0x7fc1840b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1840cf270 0x7fc18415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7efdfd2d8640 1 -- 172.21.15.32:0/3499735440 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a5970 con 0x7efdf80cc740 2023-10-16T08:16:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7efdfd2d8640 1 -- 172.21.15.32:0/3499735440 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00a5ad0 con 0x7efdf80cc740 2023-10-16T08:16:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7efdfd2d8640 1 -- 172.21.15.32:0/3499735440 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00a5e60 con 0x7efdf80cc740 2023-10-16T08:16:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 msgr2=0x7efdf80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3499735440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf80ccbf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdf80ca6a0 tx=0x7efdf001c8b0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 shutdown_connections 2023-10-16T08:16:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 0x7fc1840c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840b5580 con 0x7fc1840cf270 2023-10-16T08:16:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840b5760 con 0x7fc1840ccfb0 2023-10-16T08:16:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.258+0000 7fc18a4c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840b5a40 con 0x7fc18415eaf0 2023-10-16T08:16:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 0x7fc1840c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3499735440 >> v2:172.21.15.32:3300/0 conn(0x7efdf80cb370 0x7efdf815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 0x7fc1840c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36538/0 (socket says 172.21.15.32:36538) 2023-10-16T08:16:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 -- 172.21.15.32:0/888078799 learned_addr learned my addr 172.21.15.32:0/888078799 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc1837fe640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1840cf270 0x7fc18415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc183fff640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1840ccfb0 0x7fc1840b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3499735440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf80ccbf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 >> 172.21.15.32:0/3499735440 conn(0x7efdf806a6e0 msgr2=0x7efdf80afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 -- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1840ccfb0 msgr2=0x7fc1840b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1840ccfb0 0x7fc1840b8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 -- 172.21.15.32:0/888078799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1840cf270 msgr2=0x7fc18415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1840cf270 0x7fc18415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 -- 172.21.15.32:0/888078799 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009b30 con 0x7fc18415eaf0 2023-10-16T08:16:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc1837fe640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1840cf270 0x7fc18415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc188a66640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 0x7fc1840c6e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c430 tx=0x7fc17800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 shutdown_connections 2023-10-16T08:16:50.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7f9ad2510640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 0x7f9acc092d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7efdff2dc640 1 -- 172.21.15.32:0/3499735440 wait complete. 2023-10-16T08:16:50.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7f9ad150e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 0x7f9acc092d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7f9ad150e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 0x7f9acc092d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36546/0 (socket says 172.21.15.32:36546) 2023-10-16T08:16:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7f9ad150e640 1 -- 172.21.15.32:0/85389284 learned_addr learned my addr 172.21.15.32:0/85389284 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780052d0 con 0x7fc18415eaf0 2023-10-16T08:16:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0c3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178005430 con 0x7fc18415eaf0 2023-10-16T08:16:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178003820 con 0x7fc18415eaf0 2023-10-16T08:16:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7f9ad0d0d640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840b5d20 con 0x7fc18415eaf0 2023-10-16T08:16:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.262+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18415e130 con 0x7fc18415eaf0 2023-10-16T08:16:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9acc0927c0 0x7f9acc0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0900e0 con 0x7f9acc0ce730 2023-10-16T08:16:50.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc157bc0 con 0x7f9acc0927c0 2023-10-16T08:16:50.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc157ea0 con 0x7f9acc071f20 2023-10-16T08:16:50.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad1d0f640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9acc0927c0 0x7f9acc0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad150e640 1 -- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9acc0927c0 msgr2=0x7f9acc0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad150e640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9acc0927c0 0x7f9acc0c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad150e640 1 -- 172.21.15.32:0/85389284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 msgr2=0x7f9acc0c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad150e640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad150e640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac400b6b0 con 0x7f9acc071f20 2023-10-16T08:16:50.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad150e640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 0x7f9acc092d10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000ba50 tx=0x7f9ac000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000cc10 con 0x7f9acc071f20 2023-10-16T08:16:50.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1840b6240 con 0x7fc18415eaf0 2023-10-16T08:16:50.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac000cd70 con 0x7f9acc071f20 2023-10-16T08:16:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0014680 con 0x7f9acc071f20 2023-10-16T08:16:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc158180 con 0x7f9acc071f20 2023-10-16T08:16:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0c3760 con 0x7f9acc071f20 2023-10-16T08:16:50.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.266+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc178003980 con 0x7fc18415eaf0 2023-10-16T08:16:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7fc180ff9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1700664a0 0x7fc170068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7efdff2dc640 1 Processor -- start 2023-10-16T08:16:50.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc178083780 con 0x7fc18415eaf0 2023-10-16T08:16:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7efdff2dc640 1 -- start start 2023-10-16T08:16:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc071d20 con 0x7f9acc071f20 2023-10-16T08:16:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7fc183fff640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1700664a0 0x7fc170068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7fc183fff640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1700664a0 0x7fc170068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0ae000 tx=0x7fc17c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.270+0000 7efdff2dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf80cb370 0x7efdf80b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc1840b6240 con 0x7fc18415eaf0 2023-10-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf81f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdf8158580 0x7efdf80b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf81f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf81f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42078/0 (socket says 172.21.15.32:42078) 2023-10-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 -- 172.21.15.32:0/3685414996 learned_addr learned my addr 172.21.15.32:0/3685414996 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80c6820 con 0x7efdf80cb370 2023-10-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80c6a50 con 0x7efdf80cc740 2023-10-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf80c6d30 con 0x7efdf8158580 2023-10-16T08:16:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfe2da640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf80cb370 0x7efdf80b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfeadb640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdf8158580 0x7efdf80b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 -- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdf8158580 msgr2=0x7efdf80b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdf8158580 0x7efdf80b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 -- 172.21.15.32:0/3685414996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf80cb370 msgr2=0x7efdf80b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf80cb370 0x7efdf80b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf000d040 con 0x7efdf80cc740 2023-10-16T08:16:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdfdad9640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf81f8c00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efde800fa60 tx=0x7efde800ff20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde800c070 con 0x7efdf80cc740 2023-10-16T08:16:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde800dcd0 con 0x7efdf80cc740 2023-10-16T08:16:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf80c7010 con 0x7efdf80cc740 2023-10-16T08:16:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf80c75d0 con 0x7efdf80cc740 2023-10-16T08:16:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.274+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde801a680 con 0x7efdf80cc740 2023-10-16T08:16:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde0005260 con 0x7efdf80cc740 2023-10-16T08:16:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9ac001d3b0 con 0x7f9acc071f20 2023-10-16T08:16:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7f9ac9ffb640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ab00664a0 0x7f9ab0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ac0084420 con 0x7f9acc071f20 2023-10-16T08:16:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9ac00848e0 con 0x7f9acc071f20 2023-10-16T08:16:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7f9ad0d0d640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ab00664a0 0x7f9ab0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efde8011450 con 0x7efdf80cc740 2023-10-16T08:16:50.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7efdeeffd640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efddc0664a0 0x7efddc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:50.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.278+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efde8015070 con 0x7efdf80cc740 2023-10-16T08:16:50.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.282+0000 7f9ad0d0d640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ab00664a0 0x7f9ab0068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9acc09c440 tx=0x7f9ab8001b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.282+0000 7efdfe2da640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efddc0664a0 0x7efddc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.282+0000 7efdfe2da640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efddc0664a0 0x7efddc068960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000f70 tx=0x7efdf4007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.282+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efde8051a10 con 0x7efdf80cc740 2023-10-16T08:16:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.546+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe504164590 con 0x7fe5040c2370 2023-10-16T08:16:50.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.554+0000 7fe4faffd640 1 -- 172.21.15.32:0/1524852108 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4fc01a020 con 0x7fe5040c2370 2023-10-16T08:16:50.559 INFO:teuthology.orchestra.run.smithi032.stdout:55834574858 2023-10-16T08:16:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7c4005510 con 0x7fa7dc038fc0 2023-10-16T08:16:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe4e8066530 msgr2=0x7fe4e80689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe4e8066530 0x7fe4e80689f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5040c88e0 tx=0x7fe4f4009040 comp rx=0 tx=0).stop 2023-10-16T08:16:50.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 msgr2=0x7fe5040b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 0x7fe5040b3690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00cdf0 tx=0x7fe4fc00a710 comp rx=0 tx=0).stop 2023-10-16T08:16:50.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 shutdown_connections 2023-10-16T08:16:50.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe4e8066530 0x7fe4e80689f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5040b3bd0 0x7fe5040c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5040c2370 0x7fe5040b3690 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 --2- 172.21.15.32:0/1524852108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4e40052b0 0x7fe5041609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 >> 172.21.15.32:0/1524852108 conn(0x7fe50406a4f0 msgr2=0x7fe50406fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 shutdown_connections 2023-10-16T08:16:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.558+0000 7fe4f8ff9640 1 -- 172.21.15.32:0/1524852108 wait complete. 2023-10-16T08:16:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.562+0000 7fa7d9ffb640 1 -- 172.21.15.32:0/370810164 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7e416cbd0 con 0x7fa7dc038fc0 2023-10-16T08:16:50.566 INFO:teuthology.orchestra.run.smithi032.stdout:55834574856 2023-10-16T08:16:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.570+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa7c8066560 msgr2=0x7fa7c8068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.570+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa7c8066560 0x7fa7c8068a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0482b0 tx=0x7fa7e0008040 comp rx=0 tx=0).stop 2023-10-16T08:16:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.570+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 msgr2=0x7fa7dc0d63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.570+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 0x7fa7dc0d63e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40697b0 tx=0x7fa7e4097640 comp rx=0 tx=0).stop 2023-10-16T08:16:50.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 shutdown_connections 2023-10-16T08:16:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7dc0dff30 0x7fa7dc0353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7dc050870 0x7fa7dc034e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa7c8066560 0x7fa7c8068a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 --2- 172.21.15.32:0/370810164 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa7dc038fc0 0x7fa7dc0d63e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 >> 172.21.15.32:0/370810164 conn(0x7fa7dc054c20 msgr2=0x7fa7dc0e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 shutdown_connections 2023-10-16T08:16:50.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.574+0000 7fa7ea0ad640 1 -- 172.21.15.32:0/370810164 wait complete. 2023-10-16T08:16:50.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.582+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0e44159c60 con 0x7f0e4406c070 2023-10-16T08:16:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.582+0000 7f245ae17640 1 -- 172.21.15.32:0/1977421307 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f24540b4290 con 0x7f24540ad6c0 2023-10-16T08:16:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.582+0000 7f244a7fc640 1 -- 172.21.15.32:0/1977421307 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2450054ad0 con 0x7f24540ad6c0 2023-10-16T08:16:50.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.586+0000 7f0e3affd640 1 -- 172.21.15.32:0/3751035953 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e44159c60 con 0x7f0e4406c070 2023-10-16T08:16:50.593 INFO:teuthology.orchestra.run.smithi032.stdout:55834574858 2023-10-16T08:16:50.593 INFO:teuthology.orchestra.run.smithi032.stdout:55834574857 2023-10-16T08:16:50.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 -- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f24380664a0 msgr2=0x7f2438068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f24380664a0 0x7f2438068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24540c9510 tx=0x7f2444008040 comp rx=0 tx=0).stop 2023-10-16T08:16:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 -- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24540ad6c0 msgr2=0x7f245416b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24540ad6c0 0x7f245416b8e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f245000e8a0 tx=0x7f245000ed60 comp rx=0 tx=0).stop 2023-10-16T08:16:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 -- 172.21.15.32:0/1977421307 shutdown_connections 2023-10-16T08:16:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24540b03d0 0x7f2454151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24540afff0 0x7f245416c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f24380664a0 0x7f2438068960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 --2- 172.21.15.32:0/1977421307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f24540ad6c0 0x7f245416b8e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 -- 172.21.15.32:0/1977421307 >> 172.21.15.32:0/1977421307 conn(0x7f24540cad90 msgr2=0x7f24540cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 -- 172.21.15.32:0/1977421307 shutdown_connections 2023-10-16T08:16:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e28066570 msgr2=0x7f0e28068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e28066570 0x7f0e28068a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e440c4f30 tx=0x7f0e3c009040 comp rx=0 tx=0).stop 2023-10-16T08:16:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 msgr2=0x7f0e44144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 0x7f0e44144c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b4040 tx=0x7f0e4001d6b0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f2437fff640 1 -- 172.21.15.32:0/1977421307 wait complete. 2023-10-16T08:16:50.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.0 2023-10-16T08:16:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 shutdown_connections 2023-10-16T08:16:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e28066570 0x7f0e28068a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e4415d870 0x7f0e44147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e44159520 0x7f0e4414aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.590+0000 7f0e4b0dc640 1 --2- 172.21.15.32:0/3751035953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e4406c070 0x7f0e44144c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.594+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 >> 172.21.15.32:0/3751035953 conn(0x7f0e4406a4f0 msgr2=0x7f0e44070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.594+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 shutdown_connections 2023-10-16T08:16:50.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.594+0000 7f0e4b0dc640 1 -- 172.21.15.32:0/3751035953 wait complete. 2023-10-16T08:16:50.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.1 2023-10-16T08:16:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.614+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7efde0005510 con 0x7efdf80cc740 2023-10-16T08:16:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdeeffd640 1 -- 172.21.15.32:0/3685414996 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efde8055030 con 0x7efdf80cc740 2023-10-16T08:16:50.842 INFO:teuthology.orchestra.run.smithi032.stdout:55834574857 2023-10-16T08:16:50.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.6 2023-10-16T08:16:50.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efddc0664a0 msgr2=0x7efddc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efddc0664a0 0x7efddc068960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000f70 tx=0x7efdf4007c10 comp rx=0 tx=0).stop 2023-10-16T08:16:50.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 msgr2=0x7efdf81f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf81f8c00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efde800fa60 tx=0x7efde800ff20 comp rx=0 tx=0).stop 2023-10-16T08:16:50.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 shutdown_connections 2023-10-16T08:16:50.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efddc0664a0 0x7efddc068960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efdf8158580 0x7efdf80b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdf80cc740 0x7efdf81f8c00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 --2- 172.21.15.32:0/3685414996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf80cb370 0x7efdf80b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 >> 172.21.15.32:0/3685414996 conn(0x7efdf806a6e0 msgr2=0x7efdf8070340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 shutdown_connections 2023-10-16T08:16:50.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.618+0000 7efdff2dc640 1 -- 172.21.15.32:0/3685414996 wait complete. 2023-10-16T08:16:50.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.5 2023-10-16T08:16:50.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.7 2023-10-16T08:16:50.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd82c005510 con 0x7fd84c0c1920 2023-10-16T08:16:50.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd848ff9640 1 -- 172.21.15.32:0/4180101896 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd83c053f20 con 0x7fd84c0c1920 2023-10-16T08:16:50.849 INFO:teuthology.orchestra.run.smithi032.stdout:55834574858 2023-10-16T08:16:50.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9acc157fe0 con 0x7f9acc071f20 2023-10-16T08:16:50.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7f9ac9ffb640 1 -- 172.21.15.32:0/85389284 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9acc157fe0 con 0x7f9acc071f20 2023-10-16T08:16:50.850 INFO:teuthology.orchestra.run.smithi032.stdout:55834574857 2023-10-16T08:16:50.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd838066570 msgr2=0x7fd838068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd838066570 0x7fd838068a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd844001be0 tx=0x7fd844007c30 comp rx=0 tx=0).stop 2023-10-16T08:16:50.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 msgr2=0x7fd84c0c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 0x7fd84c0c7890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00c4d0 tx=0x7fd83c00c990 comp rx=0 tx=0).stop 2023-10-16T08:16:50.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 shutdown_connections 2023-10-16T08:16:50.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd84c160b50 0x7fd84c0c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd84c0c1920 0x7fd84c0c7890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd838066570 0x7fd838068a30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 --2- 172.21.15.32:0/4180101896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd84c0b04a0 0x7fd84c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 >> 172.21.15.32:0/4180101896 conn(0x7fd84c0cad90 msgr2=0x7fd84c162c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 shutdown_connections 2023-10-16T08:16:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.666+0000 7fd8517a7640 1 -- 172.21.15.32:0/4180101896 wait complete. 2023-10-16T08:16:50.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ab00664a0 msgr2=0x7f9ab0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ab00664a0 0x7f9ab0068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9acc09c440 tx=0x7f9ab8001b50 comp rx=0 tx=0).stop 2023-10-16T08:16:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 msgr2=0x7f9acc092d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 0x7f9acc092d10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000ba50 tx=0x7f9ac000bf10 comp rx=0 tx=0).stop 2023-10-16T08:16:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 shutdown_connections 2023-10-16T08:16:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9acc0927c0 0x7f9acc0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc0ce730 0x7f9acc0c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ab00664a0 0x7f9ab0068960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 --2- 172.21.15.32:0/85389284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9acc071f20 0x7f9acc092d10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 >> 172.21.15.32:0/85389284 conn(0x7f9acc071400 msgr2=0x7f9acc06c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 shutdown_connections 2023-10-16T08:16:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.670+0000 7f9ad2510640 1 -- 172.21.15.32:0/85389284 wait complete. 2023-10-16T08:16:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.686+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc1840b6240 con 0x7fc18415eaf0 2023-10-16T08:16:50.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.686+0000 7fc180ff9640 1 -- 172.21.15.32:0/888078799 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc178017020 con 0x7fc18415eaf0 2023-10-16T08:16:50.858 INFO:teuthology.orchestra.run.smithi032.stdout:55834574858 2023-10-16T08:16:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1700664a0 msgr2=0x7fc170068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1700664a0 0x7fc170068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0ae000 tx=0x7fc17c0a7070 comp rx=0 tx=0).stop 2023-10-16T08:16:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 msgr2=0x7fc1840c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 0x7fc1840c6e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c430 tx=0x7fc17800c8f0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 shutdown_connections 2023-10-16T08:16:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1700664a0 0x7fc170068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc18415eaf0 0x7fc1840c6e40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1840cf270 0x7fc18415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 --2- 172.21.15.32:0/888078799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1840ccfb0 0x7fc1840b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 >> 172.21.15.32:0/888078799 conn(0x7fc1840b4470 msgr2=0x7fc1840d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:50.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 shutdown_connections 2023-10-16T08:16:50.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:50.690+0000 7fc18a4c9640 1 -- 172.21.15.32:0/888078799 wait complete. 2023-10-16T08:16:50.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.4 2023-10-16T08:16:50.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.2 2023-10-16T08:16:50.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.3 2023-10-16T08:16:51.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T08:16:51.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T08:16:51.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T08:16:51.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T08:16:51.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T08:16:51.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T08:16:51.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T08:16:51.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T08:16:52.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.054+0000 7fc4ad53b640 1 Processor -- start 2023-10-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.054+0000 7fc4ad53b640 1 -- start start 2023-10-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.054+0000 7fc4ad53b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc4a8071e30 0x7fc4a80ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.054+0000 7fc4ad53b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 0x7fc4a80caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.054+0000 7fc4ad53b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4a77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 0x7fc4a80caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4a77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 0x7fc4a80caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59308/0 (socket says 172.21.15.32:59308) 2023-10-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4a77fe640 1 -- 172.21.15.32:0/2817370557 learned_addr learned my addr 172.21.15.32:0/2817370557 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a80cb470 con 0x7fc4a80c9a80 2023-10-16T08:16:52.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/2817370557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a806d010 con 0x7fc4a8071e30 2023-10-16T08:16:52.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80c8cc0 con 0x7fc4a806c2e0 2023-10-16T08:16:52.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/2817370557 >> v2:172.21.15.186:3300/0 conn(0x7fc4a8071e30 0x7fc4a80ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4acd3a640 1 -- 172.21.15.32:0/2817370557 >> v2:172.21.15.186:3300/0 conn(0x7fc4a8071e30 msgr2=0x7fc4a80ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/2817370557 >> v2:172.21.15.186:3300/0 conn(0x7fc4a8071e30 0x7fc4a80ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4acd3a640 1 -- 172.21.15.32:0/2817370557 >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 msgr2=0x7fc4a80caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/2817370557 >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 0x7fc4a80caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4acd3a640 1 -- 172.21.15.32:0/2817370557 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a80c8fa0 con 0x7fc4a806c2e0 2023-10-16T08:16:52.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.058+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/2817370557 >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 0x7fc4a80caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/2817370557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a806c6f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80b0df0 tx=0x7fc49c01e740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4718c394393e944e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4a6ffd640 1 -- 172.21.15.32:0/2817370557 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0a77d0 con 0x7fc4a806c2e0 2023-10-16T08:16:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4a6ffd640 1 -- 172.21.15.32:0/2817370557 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c0a7930 con 0x7fc4a806c2e0 2023-10-16T08:16:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4a6ffd640 1 -- 172.21.15.32:0/2817370557 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0a7cc0 con 0x7fc4a806c2e0 2023-10-16T08:16:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 msgr2=0x7fc4a806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/2817370557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a806c6f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80b0df0 tx=0x7fc49c01e740 comp rx=0 tx=0).stop 2023-10-16T08:16:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 shutdown_connections 2023-10-16T08:16:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/2817370557 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a806c6f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/2817370557 >> v2:172.21.15.32:3300/0 conn(0x7fc4a80c9a80 0x7fc4a80caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/2817370557 >> v2:172.21.15.186:3300/0 conn(0x7fc4a8071e30 0x7fc4a80ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 >> 172.21.15.32:0/2817370557 conn(0x7fc4a80afe60 msgr2=0x7fc4a80b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 shutdown_connections 2023-10-16T08:16:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.070+0000 7fc4ad53b640 1 -- 172.21.15.32:0/2817370557 wait complete. 2023-10-16T08:16:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.074+0000 7fc4ad53b640 1 Processor -- start 2023-10-16T08:16:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.074+0000 7fc4ad53b640 1 -- start start 2023-10-16T08:16:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4ad53b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a81523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4ad53b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a8071e30 0x7fc4a80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a81523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a81523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36570/0 (socket says 172.21.15.32:36570) 2023-10-16T08:16:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4a7fff640 1 -- 172.21.15.32:0/4098970768 learned_addr learned my addr 172.21.15.32:0/4098970768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a8071e30 0x7fc4a80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80c9a80 0x7fc4a80c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80c9a80 0x7fc4a80c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8071720 con 0x7fc4a80c9a80 2023-10-16T08:16:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8070e20 con 0x7fc4a8071e30 2023-10-16T08:16:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.078+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a815b3f0 con 0x7fc4a806c2e0 2023-10-16T08:16:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4acd3a640 1 -- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 msgr2=0x7fc4a81523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a81523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4acd3a640 1 -- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a8071e30 msgr2=0x7fc4a80708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a8071e30 0x7fc4a80708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4acd3a640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0007710 con 0x7fc4a80c9a80 2023-10-16T08:16:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a81523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a8071e30 0x7fc4a80708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4acd3a640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80c9a80 0x7fc4a80c75d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80c8e00 tx=0x7fc49c01b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0af640 con 0x7fc4a80c9a80 2023-10-16T08:16:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00c040 con 0x7fc4a80c9a80 2023-10-16T08:16:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0af9e0 con 0x7fc4a80c9a80 2023-10-16T08:16:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c009420 con 0x7fc4a80c9a80 2023-10-16T08:16:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.082+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a80c22e0 con 0x7fc4a80c9a80 2023-10-16T08:16:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.086+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc488005260 con 0x7fc4a80c9a80 2023-10-16T08:16:52.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.086+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc49c0afb40 con 0x7fc4a80c9a80 2023-10-16T08:16:52.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.090+0000 7fc4a4ff9640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4940664f0 0x7fc4940689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.090+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc49c11d560 con 0x7fc4a80c9a80 2023-10-16T08:16:52.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.090+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4940664f0 0x7fc4940689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.090+0000 7f14a04ea640 1 Processor -- start 2023-10-16T08:16:52.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.090+0000 7f14a04ea640 1 -- start start 2023-10-16T08:16:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.090+0000 7f14a04ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14980d15d0 0x7f14980cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149e286640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f149815cb90 0x7f14980d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37412/0 (socket says 172.21.15.32:37412) 2023-10-16T08:16:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149ea87640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14980d3010 0x7f14980c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44508/0 (socket says 172.21.15.32:44508) 2023-10-16T08:16:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149ea87640 1 -- 172.21.15.32:0/330952665 learned_addr learned my addr 172.21.15.32:0/330952665 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980ce770 con 0x7f14980d3010 2023-10-16T08:16:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4940664f0 0x7fc4940689b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc498012500 tx=0x7fc49800a740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980d0600 con 0x7f149815cb90 2023-10-16T08:16:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14980c6450 con 0x7f14980d15d0 2023-10-16T08:16:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149da85640 1 --2- 172.21.15.32:0/330952665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14980d15d0 0x7f14980cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149da85640 1 -- 172.21.15.32:0/330952665 >> v1:172.21.15.186:6789/0 conn(0x7f149815cb90 legacy=0x7f14980d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149da85640 1 -- 172.21.15.32:0/330952665 >> v1:172.21.15.32:6789/0 conn(0x7f14980d3010 legacy=0x7f14980c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f149da85640 1 -- 172.21.15.32:0/330952665 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980d08a0 con 0x7f14980d15d0 2023-10-16T08:16:52.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f8e9a0f7640 1 Processor -- start 2023-10-16T08:16:52.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.094+0000 7f8e9a0f7640 1 -- start start 2023-10-16T08:16:52.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e9a0f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e941590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e9a0f7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940ca760 con 0x7f8e940cb0e0 2023-10-16T08:16:52.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc49c0ea820 con 0x7fc4a80c9a80 2023-10-16T08:16:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e9a0f7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e94071060 con 0x7f8e940c9900 2023-10-16T08:16:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e9a0f7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c9ce0 con 0x7f8e940ada90 2023-10-16T08:16:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e92ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e941590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e92ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e941590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36582/0 (socket says 172.21.15.32:36582) 2023-10-16T08:16:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e92ffd640 1 -- 172.21.15.32:0/1578506285 learned_addr learned my addr 172.21.15.32:0/1578506285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f7c5ee46640 1 Processor -- start 2023-10-16T08:16:52.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e92ffd640 1 -- 172.21.15.32:0/1578506285 >> v1:172.21.15.186:6789/0 conn(0x7f8e940c9900 legacy=0x7f8e940cc230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:16:52.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f7c5ee46640 1 -- start start 2023-10-16T08:16:52.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e92ffd640 1 -- 172.21.15.32:0/1578506285 >> v1:172.21.15.32:6789/0 conn(0x7f8e940cb0e0 legacy=0x7f8e940c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8e92ffd640 1 -- 172.21.15.32:0/1578506285 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940adef0 con 0x7f8e940ada90 2023-10-16T08:16:52.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8a5dde4640 1 Processor -- start 2023-10-16T08:16:52.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.098+0000 7f8a5dde4640 1 -- start start 2023-10-16T08:16:52.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5ee46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c580cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5ee46640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c580ce770 con 0x7f7c580d15d0 2023-10-16T08:16:52.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5ee46640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c580d0600 con 0x7f7c5815cb90 2023-10-16T08:16:52.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5ee46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580c6450 con 0x7f7c580d3010 2023-10-16T08:16:52.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a5dde4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a580afd90 0x7f8a580718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a5dde4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580c97c0 con 0x7f8a5806c070 2023-10-16T08:16:52.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a5dde4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5806ffa0 con 0x7f8a580c8fe0 2023-10-16T08:16:52.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a5dde4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580cbd60 con 0x7f8a580afd90 2023-10-16T08:16:52.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f149da85640 1 --2- 172.21.15.32:0/330952665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14980d15d0 0x7f14980cfb10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f148c0097f0 tx=0x7f148c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb625fea818786af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5cbe2640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f7c5815cb90 0x7f7c580d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37434/0 (socket says 172.21.15.32:37434) 2023-10-16T08:16:52.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5cbe2640 1 -- 172.21.15.32:0/494411981 learned_addr learned my addr 172.21.15.32:0/494411981 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:52.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a57fff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f8a580c8fe0 0x7f8a580b0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37444/0 (socket says 172.21.15.32:37444) 2023-10-16T08:16:52.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a57fff640 1 -- 172.21.15.32:0/2356979864 learned_addr learned my addr 172.21.15.32:0/2356979864 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:52.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c5d3e3640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c580cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/2356979864 >> v2:172.21.15.32:3300/0 conn(0x7f8a580afd90 0x7f8a580718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341072880 0 0) 0x7f7c580d0600 con 0x7f7c5815cb90 2023-10-16T08:16:52.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c48003530 con 0x7f7c5815cb90 2023-10-16T08:16:52.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227750563 0 0) 0x7f8a5806ffa0 con 0x7f8a580c8fe0 2023-10-16T08:16:52.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f149d284640 1 -- 172.21.15.32:0/330952665 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0ab600 con 0x7f14980d15d0 2023-10-16T08:16:52.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a48003450 con 0x7f8a580c8fe0 2023-10-16T08:16:52.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f149d284640 1 -- 172.21.15.32:0/330952665 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c0ab760 con 0x7f14980d15d0 2023-10-16T08:16:52.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434206682 0 0) 0x7f7c48003530 con 0x7f7c5815cb90 2023-10-16T08:16:52.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 msgr2=0x7f7c580cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c580cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 >> v1:172.21.15.32:6789/0 conn(0x7f7c580d15d0 legacy=0x7f7c580c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580d08a0 con 0x7f7c5815cb90 2023-10-16T08:16:52.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f7c4c0050a0 con 0x7f7c5815cb90 2023-10-16T08:16:52.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 >> v1:172.21.15.186:6789/0 conn(0x7f7c5815cb90 legacy=0x7f7c580d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c48003980 0x7f7c48003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c48004330 0x7f7c48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c48005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580c6450 con 0x7f7c48003980 2023-10-16T08:16:52.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c48014250 con 0x7f7c48004330 2023-10-16T08:16:52.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c480143e0 con 0x7f7c580d3010 2023-10-16T08:16:52.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f149d284640 1 -- 172.21.15.32:0/330952665 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0ab600 con 0x7f14980d15d0 2023-10-16T08:16:52.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3960465479 0 0) 0x7f8a48003450 con 0x7f8a580c8fe0 2023-10-16T08:16:52.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 >> v2:172.21.15.32:3300/0 conn(0x7f8a580afd90 msgr2=0x7f8a580718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 --2- 172.21.15.32:0/2356979864 >> v2:172.21.15.32:3300/0 conn(0x7f8a580afd90 0x7f8a580718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 >> v1:172.21.15.32:6789/0 conn(0x7f8a5806c070 legacy=0x7f8a5806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 --> v1:172.21.15.186:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5816d350 con 0x7f8a580c8fe0 2023-10-16T08:16:52.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 <== mon.1 v1:172.21.15.186:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f8a44004ce0 con 0x7f8a580c8fe0 2023-10-16T08:16:52.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 >> v1:172.21.15.186:6789/0 conn(0x7f8a580c8fe0 legacy=0x7f8a580b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48003b50 0x7f8a48003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48006710 0x7f8a48008ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a48009440 0x7f8a480098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580cbd60 con 0x7f8a48003b50 2023-10-16T08:16:52.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4800a1d0 con 0x7f8a48009440 2023-10-16T08:16:52.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4800a390 con 0x7f8a48006710 2023-10-16T08:16:52.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.102+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/2356979864 >> v2:172.21.15.32:3300/0 conn(0x7f8a580afd90 0x7f8a580718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a57fff640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48006710 0x7f8a48008ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e92ffd640 1 --2- 172.21.15.32:0/1578506285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e941590e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e880098a0 tx=0x7f8e8801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bfd859292a49cec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5d3e3640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c48005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c57fff640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c48004330 0x7f7c48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14980d15d0 msgr2=0x7f14980cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 --2- 172.21.15.32:0/330952665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14980d15d0 0x7f14980cfb10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f148c0097f0 tx=0x7f148c01c420 comp rx=0 tx=0).stop 2023-10-16T08:16:52.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5d5e3640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a48009440 0x7f8a480098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48003b50 0x7f8a48003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5d3e3640 1 -- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c48004330 msgr2=0x7f7c48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5d3e3640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c48004330 0x7f7c48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5cbe2640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c48003980 0x7f7c48003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5d3e3640 1 -- 172.21.15.32:0/494411981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c48003980 msgr2=0x7f7c48003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5d3e3640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c48003980 0x7f7c48003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5d3e3640 1 -- 172.21.15.32:0/494411981 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580d08a0 con 0x7f7c580d3010 2023-10-16T08:16:52.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e927fc640 1 -- 172.21.15.32:0/1578506285 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880ad680 con 0x7f8e940ada90 2023-10-16T08:16:52.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5cde2640 1 -- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48006710 msgr2=0x7f8a48008ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48006710 0x7f8a48008ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5cde2640 1 -- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a48009440 msgr2=0x7f8a480098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a48009440 0x7f8a480098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 shutdown_connections 2023-10-16T08:16:52.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 --2- 172.21.15.32:0/330952665 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14980d15d0 0x7f14980cfb10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 >> 172.21.15.32:0/330952665 conn(0x7f149809bc50 msgr2=0x7f14980d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5d5e3640 1 -- 172.21.15.32:0/2356979864 reap_dead start 2023-10-16T08:16:52.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a5cde2640 1 -- 172.21.15.32:0/2356979864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5816d350 con 0x7f8a48003b50 2023-10-16T08:16:52.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 shutdown_connections 2023-10-16T08:16:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8a57fff640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48006710 0x7f8a48008ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e927fc640 1 -- 172.21.15.32:0/1578506285 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e880ad7e0 con 0x7f8e940ada90 2023-10-16T08:16:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e927fc640 1 -- 172.21.15.32:0/1578506285 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880adb70 con 0x7f8e940ada90 2023-10-16T08:16:52.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f14a04ea640 1 -- 172.21.15.32:0/330952665 wait complete. 2023-10-16T08:16:52.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1578506285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 msgr2=0x7f8e941590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1578506285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e941590e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e880098a0 tx=0x7f8e8801c490 comp rx=0 tx=0).stop 2023-10-16T08:16:52.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1578506285 shutdown_connections 2023-10-16T08:16:52.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1578506285 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e941590e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1578506285 >> 172.21.15.32:0/1578506285 conn(0x7f8e9406c190 msgr2=0x7f8e9406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1578506285 shutdown_connections 2023-10-16T08:16:52.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1578506285 wait complete. 2023-10-16T08:16:52.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.106+0000 7f7c5cbe2640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c48003980 0x7f7c48003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:52.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f14a04ea640 1 Processor -- start 2023-10-16T08:16:52.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f8e9a0f7640 1 Processor -- start 2023-10-16T08:16:52.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f14a04ea640 1 -- start start 2023-10-16T08:16:52.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48003b50 0x7f8a48003f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009c90 tx=0x7f8a4c02dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5a941e4bfd3ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c004220 con 0x7f8a48003b50 2023-10-16T08:16:52.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f8e9a0f7640 1 -- start start 2023-10-16T08:16:52.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f7c5d3e3640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c48005110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e170 tx=0x7f7c5001cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ac8122b5f5445 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c004380 con 0x7f8a48003b50 2023-10-16T08:16:52.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f8a577fe640 1 -- 172.21.15.32:0/2356979864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0b68c0 con 0x7f8a48003b50 2023-10-16T08:16:52.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500a5b50 con 0x7f7c580d3010 2023-10-16T08:16:52.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5000f040 con 0x7f7c580d3010 2023-10-16T08:16:52.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.110+0000 7f7c577fe640 1 -- 172.21.15.32:0/494411981 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500ad440 con 0x7f7c580d3010 2023-10-16T08:16:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14980d15d0 0x7f1498157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 0x7f149809c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149815cb90 0x7f14980b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149815aae0 con 0x7f14980d15d0 2023-10-16T08:16:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149815adc0 con 0x7f14980d3010 2023-10-16T08:16:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149815b0a0 con 0x7f149815cb90 2023-10-16T08:16:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149e286640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14980d15d0 0x7f1498157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 0x7f149809c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 0x7f149809c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42128/0 (socket says 172.21.15.32:42128) 2023-10-16T08:16:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 -- 172.21.15.32:0/1577434255 learned_addr learned my addr 172.21.15.32:0/1577434255 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149ea87640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149815cb90 0x7f14980b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 -- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149815cb90 msgr2=0x7f14980b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149815cb90 0x7f14980b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 -- 172.21.15.32:0/1577434255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14980d15d0 msgr2=0x7f1498157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14980d15d0 0x7f1498157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 -- 172.21.15.32:0/1577434255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009420 con 0x7f14980d3010 2023-10-16T08:16:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149e286640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14980d15d0 0x7f1498157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 -- 172.21.15.32:0/2356979864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48003b50 msgr2=0x7f8a48003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48003b50 0x7f8a48003f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009c90 tx=0x7f8a4c02dbe0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 -- 172.21.15.32:0/2356979864 shutdown_connections 2023-10-16T08:16:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 --2- 172.21.15.32:0/2356979864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48003b50 0x7f8a48003f70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 -- 172.21.15.32:0/2356979864 >> 172.21.15.32:0/2356979864 conn(0x7f8a580cad90 msgr2=0x7f8a580cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 -- 172.21.15.32:0/2356979864 shutdown_connections 2023-10-16T08:16:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8a5dde4640 1 -- 172.21.15.32:0/2356979864 wait complete. 2023-10-16T08:16:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f7c5ee46640 1 -- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 msgr2=0x7f7c48005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/494411981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c580d3010 0x7f7c48005110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e170 tx=0x7f7c5001cc50 comp rx=0 tx=0).stop 2023-10-16T08:16:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e9a0f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e940c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e9a0f7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e940c9900 0x7f8e940c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e940c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e9a0f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e940cb0e0 0x7f8e940c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e940c58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36640/0 (socket says 172.21.15.32:36640) 2023-10-16T08:16:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e92ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e940c9900 0x7f8e940c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e92ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e940c9900 0x7f8e940c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42144/0 (socket says 172.21.15.32:42144) 2023-10-16T08:16:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f7c5d3e3640 1 -- 172.21.15.32:0/494411981 reap_dead start 2023-10-16T08:16:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e940cb0e0 0x7f8e940c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f7c5ee46640 1 -- 172.21.15.32:0/494411981 shutdown_connections 2023-10-16T08:16:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 -- 172.21.15.32:0/1091473002 learned_addr learned my addr 172.21.15.32:0/1091473002 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c7d80 con 0x7f8e940cb0e0 2023-10-16T08:16:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9415b9b0 con 0x7f8e940c9900 2023-10-16T08:16:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f7c5ee46640 1 -- 172.21.15.32:0/494411981 >> 172.21.15.32:0/494411981 conn(0x7f7c5809bc50 msgr2=0x7f7c580d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c22c0 con 0x7f8e940ada90 2023-10-16T08:16:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f149da85640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 0x7f149809c6c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f148c019030 tx=0x7f148c0191c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0abad0 con 0x7f14980d3010 2023-10-16T08:16:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c0abc30 con 0x7f14980d3010 2023-10-16T08:16:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c003620 con 0x7f14980d3010 2023-10-16T08:16:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 Processor -- start 2023-10-16T08:16:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149815b380 con 0x7f14980d3010 2023-10-16T08:16:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f7c5ee46640 1 -- 172.21.15.32:0/494411981 shutdown_connections 2023-10-16T08:16:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f7c5ee46640 1 -- 172.21.15.32:0/494411981 wait complete. 2023-10-16T08:16:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1498157ed0 con 0x7f14980d3010 2023-10-16T08:16:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 -- start start 2023-10-16T08:16:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 -- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e940c9900 msgr2=0x7f8e940c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e940c9900 0x7f8e940c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 -- 172.21.15.32:0/1091473002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e940cb0e0 msgr2=0x7f8e940c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 0x7f8a580888d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a580c4540 0x7f8a580c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 0x7f8a580c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c3860 con 0x7f8a580c4540 2023-10-16T08:16:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c4ee0 con 0x7f8a580c1e90 2023-10-16T08:16:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5dde4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c2be0 con 0x7f8a48003b50 2023-10-16T08:16:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5cde2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 0x7f8a580888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5cde2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 0x7f8a580888d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36644/0 (socket says 172.21.15.32:36644) 2023-10-16T08:16:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5d5e3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 0x7f8a580c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5d5e3640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 0x7f8a580c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42160/0 (socket says 172.21.15.32:42160) 2023-10-16T08:16:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a5cde2640 1 -- 172.21.15.32:0/970056919 learned_addr learned my addr 172.21.15.32:0/970056919 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8a57fff640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a580c4540 0x7f8a580c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5d5e3640 1 -- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 msgr2=0x7f8a580888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5d5e3640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 0x7f8a580888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5d5e3640 1 -- 172.21.15.32:0/970056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a580c4540 msgr2=0x7f8a580c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5d5e3640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a580c4540 0x7f8a580c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5d5e3640 1 -- 172.21.15.32:0/970056919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580c2ec0 con 0x7f8a580c1e90 2023-10-16T08:16:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 0x7f8a580888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a57fff640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a580c4540 0x7f8a580c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e940cb0e0 0x7f8e940c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880094d0 con 0x7f8e940ada90 2023-10-16T08:16:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.114+0000 7f8e937fe640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e940c58f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400b4a0 tx=0x7f8e8400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84010610 con 0x7f8e940ada90 2023-10-16T08:16:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e84010790 con 0x7f8e940ada90 2023-10-16T08:16:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84010b40 con 0x7f8e940ada90 2023-10-16T08:16:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e940c25a0 con 0x7f8e940ada90 2023-10-16T08:16:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e940c2b60 con 0x7f8e940ada90 2023-10-16T08:16:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5d5e3640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 0x7f8a580c2310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a580cb9d0 tx=0x7f8a5000fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50003b90 con 0x7f8a580c1e90 2023-10-16T08:16:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50003d10 con 0x7f8a580c1e90 2023-10-16T08:16:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50005310 con 0x7f8a580c1e90 2023-10-16T08:16:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f7c5ee46640 1 Processor -- start 2023-10-16T08:16:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f7c5ee46640 1 -- start start 2023-10-16T08:16:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.118+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e940c2d50 con 0x7f8e940ada90 2023-10-16T08:16:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1480005260 con 0x7f14980d3010 2023-10-16T08:16:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5dde4640 1 -- 172.21.15.32:0/970056919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580c31a0 con 0x7f8a580c1e90 2023-10-16T08:16:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8a5dde4640 1 -- 172.21.15.32:0/970056919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a58088e10 con 0x7f8a580c1e90 2023-10-16T08:16:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.122+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8e84010ca0 con 0x7f8e940ada90 2023-10-16T08:16:52.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30005260 con 0x7f8a580c1e90 2023-10-16T08:16:52.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f8e7bfff640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e80077580 0x7f8e80079a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e84083e40 con 0x7f8e940ada90 2023-10-16T08:16:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5ee46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 0x7f7c580cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5ee46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c5814d910 0x7f7c580b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 0x7f7c580cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 0x7f7c580cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42170/0 (socket says 172.21.15.32:42170) 2023-10-16T08:16:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 -- 172.21.15.32:0/3579764433 learned_addr learned my addr 172.21.15.32:0/3579764433 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8a50005470 con 0x7f8a580c1e90 2023-10-16T08:16:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f148c003780 con 0x7f14980d3010 2023-10-16T08:16:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f8e92ffd640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e80077580 0x7f8e80079a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c57fff640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c5814d910 0x7f7c580b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c581f0210 0x7f7c581f0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581f0bd0 con 0x7f7c581f0210 2023-10-16T08:16:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5814e2d0 con 0x7f7c580cba80 2023-10-16T08:16:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580ca480 con 0x7f7c5814d910 2023-10-16T08:16:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5d3e3640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c581f0210 0x7f7c581f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 -- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c5814d910 msgr2=0x7f7c580b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c5814d910 0x7f7c580b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 -- 172.21.15.32:0/3579764433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c581f0210 msgr2=0x7f7c581f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c581f0210 0x7f7c581f0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.126+0000 7f7c5cbe2640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580ca720 con 0x7f7c580cba80 2023-10-16T08:16:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f7c5cbe2640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 0x7f7c580cbe60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00d830 tx=0x7f7c4c00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f148affd640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f147c066570 0x7f147c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f148c11e520 con 0x7f14980d3010 2023-10-16T08:16:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f149e286640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f147c066570 0x7f147c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f8a557fa640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8a3c0664a0 0x7f8a3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a50083fe0 con 0x7f8a580c1e90 2023-10-16T08:16:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8a3c0664a0 0x7f8a3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f149e286640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f147c066570 0x7f147c068a30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1490007fd0 tx=0x7f1490009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f8e92ffd640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e80077580 0x7f8e80079a40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8e880af000 tx=0x7f8e880a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8e84051060 con 0x7f8e940ada90 2023-10-16T08:16:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.130+0000 7f8a5cde2640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8a3c0664a0 0x7f8a3c068960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0c0040 tx=0x7f8a4c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.134+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8a50051210 con 0x7f8a580c1e90 2023-10-16T08:16:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.138+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f148c0eb720 con 0x7f14980d3010 2023-10-16T08:16:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.138+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00b6b0 con 0x7f7c580cba80 2023-10-16T08:16:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.138+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c00b810 con 0x7f7c580cba80 2023-10-16T08:16:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.138+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00bbc0 con 0x7f7c580cba80 2023-10-16T08:16:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.138+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c580caa00 con 0x7f7c580cba80 2023-10-16T08:16:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.138+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c580b8500 con 0x7f7c580cba80 2023-10-16T08:16:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.142+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c4c00bd20 con 0x7f7c580cba80 2023-10-16T08:16:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.142+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c58074010 con 0x7f7c580cba80 2023-10-16T08:16:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.146+0000 7f7c557fa640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7c3c0665a0 0x7f7c3c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.146+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7c4c0843f0 con 0x7f7c580cba80 2023-10-16T08:16:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.146+0000 7f7c57fff640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7c3c0665a0 0x7f7c3c068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.146+0000 7f7c57fff640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7c3c0665a0 0x7f7c3c068a60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806b2a0 tx=0x7f7c44007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.150+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c4c051560 con 0x7f7c580cba80 2023-10-16T08:16:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.150+0000 7fc08cd65640 1 Processor -- start 2023-10-16T08:16:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.150+0000 7fc08cd65640 1 -- start start 2023-10-16T08:16:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08cd65640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc08815cb90 0x7fc0880cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08cd65640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0880d3010 0x7fc0880d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc08815cb90 0x7fc0880cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc08815cb90 0x7fc0880cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42186/0 (socket says 172.21.15.32:42186) 2023-10-16T08:16:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 -- 172.21.15.32:0/948559215 learned_addr learned my addr 172.21.15.32:0/948559215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0880d1850 con 0x7fc0880d1400 2023-10-16T08:16:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0880ce770 con 0x7fc0880d3010 2023-10-16T08:16:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0880d0600 con 0x7fc08815cb90 2023-10-16T08:16:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08559a640 1 -- 172.21.15.32:0/948559215 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165400848 0 0) 0x7fc0800014a0 con 0x7fc0880d1400 2023-10-16T08:16:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08559a640 1 -- 172.21.15.32:0/948559215 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc070003530 con 0x7fc0880d1400 2023-10-16T08:16:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc085d9b640 1 --2- 172.21.15.32:0/948559215 >> v2:172.21.15.32:3300/0 conn(0x7fc0880d3010 0x7fc0880d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 -- 172.21.15.32:0/948559215 >> v2:172.21.15.32:3300/0 conn(0x7fc0880d3010 msgr2=0x7fc0880d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08559a640 1 -- 172.21.15.32:0/948559215 <== mon.1 v1:172.21.15.186:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675300113 0 0) 0x7fc070003530 con 0x7fc0880d1400 2023-10-16T08:16:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 --2- 172.21.15.32:0/948559215 >> v2:172.21.15.32:3300/0 conn(0x7fc0880d3010 0x7fc0880d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 -- 172.21.15.32:0/948559215 >> v1:172.21.15.186:6789/0 conn(0x7fc0880d1400 legacy=0x7fc0880cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc08659c640 1 -- 172.21.15.32:0/948559215 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0880c6450 con 0x7fc08815cb90 2023-10-16T08:16:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7feeea965640 1 Processor -- start 2023-10-16T08:16:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7fc085d9b640 1 --2- 172.21.15.32:0/948559215 >> v2:172.21.15.32:3300/0 conn(0x7fc0880d3010 0x7fc0880d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.154+0000 7feeea965640 1 -- start start 2023-10-16T08:16:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feeea965640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feee40d1410 0x7feee4167960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feeea965640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40d1930 con 0x7feee40ce0b0 2023-10-16T08:16:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feeea965640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40d1c10 con 0x7feee415cf00 2023-10-16T08:16:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feeea965640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee415d2e0 con 0x7feee40d1410 2023-10-16T08:16:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee8f02640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feee40d1410 0x7feee4167960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee8f02640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feee40d1410 0x7feee4167960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59392/0 (socket says 172.21.15.32:59392) 2023-10-16T08:16:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee8f02640 1 -- 172.21.15.32:0/499595851 learned_addr learned my addr 172.21.15.32:0/499595851 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7fc08659c640 1 --2- 172.21.15.32:0/948559215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc08815cb90 0x7fc0880cfb10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc08808ba70 tx=0x7fc07801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6d62a84f1e9659 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee2ffd640 1 -- 172.21.15.32:0/499595851 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845944097 0 0) 0x7feee40d1c10 con 0x7feee415cf00 2023-10-16T08:16:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee2ffd640 1 -- 172.21.15.32:0/499595851 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feecc003530 con 0x7feee415cf00 2023-10-16T08:16:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee8f02640 1 -- 172.21.15.32:0/499595851 >> v1:172.21.15.186:6789/0 conn(0x7feee415cf00 legacy=0x7feee40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee8f02640 1 -- 172.21.15.32:0/499595851 >> v1:172.21.15.32:6789/0 conn(0x7feee40ce0b0 legacy=0x7feee40d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7feee8f02640 1 -- 172.21.15.32:0/499595851 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40d1ef0 con 0x7feee40d1410 2023-10-16T08:16:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.158+0000 7fc08559a640 1 -- 172.21.15.32:0/948559215 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080001b90 con 0x7fc08815cb90 2023-10-16T08:16:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08559a640 1 -- 172.21.15.32:0/948559215 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0780a7050 con 0x7fc08815cb90 2023-10-16T08:16:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 Processor -- start 2023-10-16T08:16:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc08815cb90 msgr2=0x7fc0880cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 --2- 172.21.15.32:0/948559215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc08815cb90 0x7fc0880cfb10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc08808ba70 tx=0x7fc07801c380 comp rx=0 tx=0).stop 2023-10-16T08:16:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 shutdown_connections 2023-10-16T08:16:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 --2- 172.21.15.32:0/948559215 >> v2:172.21.15.32:3300/0 conn(0x7fc0880d3010 0x7fc0880d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 --2- 172.21.15.32:0/948559215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc08815cb90 0x7fc0880cfb10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 >> 172.21.15.32:0/948559215 conn(0x7fc08809bc50 msgr2=0x7fc0880d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 -- start start 2023-10-16T08:16:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77c8164520 0x7f77c80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f77c80ad960 0x7f77c80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c80c96f0 con 0x7f77c80c9b80 2023-10-16T08:16:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c8164900 con 0x7f77c8164520 2023-10-16T08:16:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77ccc54640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c80cb950 con 0x7f77c80ad960 2023-10-16T08:16:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77c6d9d640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f77c80c9b80 0x7f77c8164ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37468/0 (socket says 172.21.15.32:37468) 2023-10-16T08:16:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7f77c6d9d640 1 -- 172.21.15.32:0/3531211853 learned_addr learned my addr 172.21.15.32:0/3531211853 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 shutdown_connections 2023-10-16T08:16:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c5d9b640 1 --2- 172.21.15.32:0/3531211853 >> v2:172.21.15.186:3300/0 conn(0x7f77c80ad960 0x7f77c80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c659c640 1 --2- 172.21.15.32:0/3531211853 >> v2:172.21.15.32:3300/0 conn(0x7f77c8164520 0x7f77c80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.162+0000 7fc08cd65640 1 -- 172.21.15.32:0/948559215 wait complete. 2023-10-16T08:16:52.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c559a640 1 -- 172.21.15.32:0/3531211853 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657901895 0 0) 0x7f77c80c96f0 con 0x7f77c80c9b80 2023-10-16T08:16:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7fc08cd65640 1 Processor -- start 2023-10-16T08:16:52.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7feee8f02640 1 --2- 172.21.15.32:0/499595851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 0x7feee4167960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feedc009e50 tx=0x7feedc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4864bad878beef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c559a640 1 -- 172.21.15.32:0/3531211853 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b0003530 con 0x7f77c80c9b80 2023-10-16T08:16:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c659c640 1 -- 172.21.15.32:0/3531211853 >> v2:172.21.15.186:3300/0 conn(0x7f77c80ad960 msgr2=0x7f77c80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c659c640 1 --2- 172.21.15.32:0/3531211853 >> v2:172.21.15.186:3300/0 conn(0x7f77c80ad960 0x7f77c80b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c659c640 1 -- 172.21.15.32:0/3531211853 >> v1:172.21.15.186:6789/0 conn(0x7f77c80c9b80 legacy=0x7f77c8164ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c659c640 1 -- 172.21.15.32:0/3531211853 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80cbed0 con 0x7f77c8164520 2023-10-16T08:16:52.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7f77c5d9b640 1 --2- 172.21.15.32:0/3531211853 >> v2:172.21.15.186:3300/0 conn(0x7f77c80ad960 0x7f77c80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7fc08cd65640 1 -- start start 2023-10-16T08:16:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7feee2ffd640 1 -- 172.21.15.32:0/499595851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0a5820 con 0x7feee40d1410 2023-10-16T08:16:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7feee2ffd640 1 -- 172.21.15.32:0/499595851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc0a5980 con 0x7feee40d1410 2023-10-16T08:16:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.166+0000 7feee2ffd640 1 -- 172.21.15.32:0/499595851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0a5820 con 0x7feee40d1410 2023-10-16T08:16:52.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08cd65640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880d1400 0x7fc0880c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 -- 172.21.15.32:0/499595851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 msgr2=0x7feee4167960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 --2- 172.21.15.32:0/499595851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 0x7feee4167960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feedc009e50 tx=0x7feedc01c940 comp rx=0 tx=0).stop 2023-10-16T08:16:52.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08cd65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0880d3010 0x7fc088201950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 -- 172.21.15.32:0/499595851 shutdown_connections 2023-10-16T08:16:52.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 --2- 172.21.15.32:0/499595851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 0x7feee4167960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 -- 172.21.15.32:0/499595851 >> 172.21.15.32:0/499595851 conn(0x7feee40b4470 msgr2=0x7feee40b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880d1400 0x7fc0880c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880d1400 0x7fc0880c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42210/0 (socket says 172.21.15.32:42210) 2023-10-16T08:16:52.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08cd65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 0x7fc0880b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 0x7fc0880b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 0x7fc0880b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59408/0 (socket says 172.21.15.32:59408) 2023-10-16T08:16:52.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08659c640 1 -- 172.21.15.32:0/3683126825 learned_addr learned my addr 172.21.15.32:0/3683126825 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc085d9b640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0880d3010 0x7fc088201950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc088201e90 con 0x7fc08815cb90 2023-10-16T08:16:52.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0880c3d10 con 0x7fc0880d1400 2023-10-16T08:16:52.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0880c3ff0 con 0x7fc0880d3010 2023-10-16T08:16:52.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 -- 172.21.15.32:0/499595851 shutdown_connections 2023-10-16T08:16:52.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7feeea965640 1 -- 172.21.15.32:0/499595851 wait complete. 2023-10-16T08:16:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 -- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0880d3010 msgr2=0x7fc088201950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0880d3010 0x7fc088201950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 -- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880d1400 msgr2=0x7fc0880c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880d1400 0x7fc0880c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009480 con 0x7fc08815cb90 2023-10-16T08:16:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7f77c659c640 1 --2- 172.21.15.32:0/3531211853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c8164520 0x7f77c80ce010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77c806e3a0 tx=0x7f77b801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f752cda1dd2d26a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.170+0000 7fc086d9d640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 0x7fc0880b5520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc08000cb90 tx=0x7fc08000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77c559a640 1 -- 172.21.15.32:0/3531211853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b80ab610 con 0x7f77c8164520 2023-10-16T08:16:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77c559a640 1 -- 172.21.15.32:0/3531211853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b80ab770 con 0x7f77c8164520 2023-10-16T08:16:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77c559a640 1 -- 172.21.15.32:0/3531211853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b80abb00 con 0x7f77c8164520 2023-10-16T08:16:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000acc0 con 0x7fc08815cb90 2023-10-16T08:16:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0880c42d0 con 0x7fc08815cb90 2023-10-16T08:16:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0880c4890 con 0x7fc08815cb90 2023-10-16T08:16:52.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 -- 172.21.15.32:0/3531211853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c8164520 msgr2=0x7f77c80ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 --2- 172.21.15.32:0/3531211853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c8164520 0x7f77c80ce010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77c806e3a0 tx=0x7f77b801c430 comp rx=0 tx=0).stop 2023-10-16T08:16:52.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7feeea965640 1 Processor -- start 2023-10-16T08:16:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 -- 172.21.15.32:0/3531211853 shutdown_connections 2023-10-16T08:16:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 --2- 172.21.15.32:0/3531211853 >> v2:172.21.15.186:3300/0 conn(0x7f77c80ad960 0x7f77c80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7feeea965640 1 -- start start 2023-10-16T08:16:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 --2- 172.21.15.32:0/3531211853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c8164520 0x7f77c80ce010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 -- 172.21.15.32:0/3531211853 >> 172.21.15.32:0/3531211853 conn(0x7f77c806c190 msgr2=0x7f77c806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 -- 172.21.15.32:0/3531211853 shutdown_connections 2023-10-16T08:16:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080007de0 con 0x7fc08815cb90 2023-10-16T08:16:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feeea965640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feee40ce0b0 0x7feee40cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feeea965640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 0x7feee415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0880c4ab0 con 0x7fc08815cb90 2023-10-16T08:16:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000acc0 con 0x7fc08815cb90 2023-10-16T08:16:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feee3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feee40ce0b0 0x7feee40cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feee3fff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feee40ce0b0 0x7feee40cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:42216/0 (socket says 172.21.15.32:42216) 2023-10-16T08:16:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feee3fff640 1 -- 172.21.15.32:0/2494750909 learned_addr learned my addr 172.21.15.32:0/2494750909 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feeea965640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feee415cf00 0x7feee4151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feeea965640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee415a940 con 0x7feee40d1410 2023-10-16T08:16:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.174+0000 7f77ccc54640 1 -- 172.21.15.32:0/3531211853 wait complete. 2023-10-16T08:16:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7f77ccc54640 1 Processor -- start 2023-10-16T08:16:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feee8f02640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feee415cf00 0x7feee4151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feeea965640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee4158d50 con 0x7feee40ce0b0 2023-10-16T08:16:52.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7f77ccc54640 1 -- start start 2023-10-16T08:16:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7feeea965640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee415ab50 con 0x7feee415cf00 2023-10-16T08:16:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77ccc54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 0x7f77c814c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77ccc54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c80c9b80 0x7f77c80c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77ccc54640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8164520 0x7f77c80c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee8f02640 1 -- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feee40ce0b0 msgr2=0x7feee40cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee8f02640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feee40ce0b0 0x7feee40cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee8f02640 1 -- 172.21.15.32:0/2494750909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 msgr2=0x7feee415b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee8f02640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 0x7feee415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee8f02640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc009b30 con 0x7feee415cf00 2023-10-16T08:16:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c659c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 0x7f77c814c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c659c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 0x7f77c814c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36676/0 (socket says 172.21.15.32:36676) 2023-10-16T08:16:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c4fb0 con 0x7f77c80c9b80 2023-10-16T08:16:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c659c640 1 -- 172.21.15.32:0/792275631 learned_addr learned my addr 172.21.15.32:0/792275631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c7470 con 0x7f77c8164520 2023-10-16T08:16:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80c7750 con 0x7f77c80ad960 2023-10-16T08:16:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee8f02640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feee415cf00 0x7feee4151370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feedc0ab000 tx=0x7feedc0196e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c6d9d640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8164520 0x7f77c80c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c6d9d640 1 -- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 msgr2=0x7f77c814c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c6d9d640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 0x7f77c814c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c6d9d640 1 -- 172.21.15.32:0/792275631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c80c9b80 msgr2=0x7f77c80c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c6d9d640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c80c9b80 0x7f77c80c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c6d9d640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc009420 con 0x7f77c8164520 2023-10-16T08:16:52.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0bd070 con 0x7feee415cf00 2023-10-16T08:16:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7f77c659c640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 0x7f77c814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc0b8740 con 0x7feee415cf00 2023-10-16T08:16:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0a5820 con 0x7feee415cf00 2023-10-16T08:16:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feeea965640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee415ae30 con 0x7feee415cf00 2023-10-16T08:16:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7feeea965640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee4159280 con 0x7feee415cf00 2023-10-16T08:16:52.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.178+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc080001990 con 0x7fc08815cb90 2023-10-16T08:16:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7fc076ffd640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0680664c0 0x7fc068068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.182+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc0800557d0 con 0x7fc08815cb90 2023-10-16T08:16:52.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7f77c6d9d640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8164520 0x7f77c80c6680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77c000caa0 tx=0x7f77c000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c000aae0 con 0x7f77c8164520 2023-10-16T08:16:52.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b8009420 con 0x7f77c8164520 2023-10-16T08:16:52.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c80c7d10 con 0x7f77c8164520 2023-10-16T08:16:52.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7fc08659c640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0680664c0 0x7fc068068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c000ac40 con 0x7f77c8164520 2023-10-16T08:16:52.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c000aae0 con 0x7f77c8164520 2023-10-16T08:16:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.186+0000 7fc08659c640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0680664c0 0x7fc068068980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc07801aa30 tx=0x7fc0780a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feedc0b88b0 con 0x7feee415cf00 2023-10-16T08:16:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f77c0007070 con 0x7f77c8164520 2023-10-16T08:16:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7f77b6ffd640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77a8066500 0x7f77a80689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7feee0ff9640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feed00664a0 0x7feed0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f77c00839d0 con 0x7f77c8164520 2023-10-16T08:16:52.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feedc11eea0 con 0x7feee415cf00 2023-10-16T08:16:52.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac005260 con 0x7f77c8164520 2023-10-16T08:16:52.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7f77c659c640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77a8066500 0x7f77a80689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7feee3fff640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feed00664a0 0x7feed0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc080051410 con 0x7fc08815cb90 2023-10-16T08:16:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7feee3fff640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feed00664a0 0x7feed0068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feed80011f0 tx=0x7feed8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.190+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4005260 con 0x7feee415cf00 2023-10-16T08:16:52.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.194+0000 7f77c659c640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77a8066500 0x7f77a80689c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77b801c470 tx=0x7f77b80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.194+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f77c0050c00 con 0x7f77c8164520 2023-10-16T08:16:52.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.198+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feedc0ebf90 con 0x7feee415cf00 2023-10-16T08:16:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.430+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc488005510 con 0x7fc4a80c9a80 2023-10-16T08:16:52.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.430+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/4098970768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc49c0ea820 con 0x7fc4a80c9a80 2023-10-16T08:16:52.436 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4940664f0 msgr2=0x7fc4940689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4940664f0 0x7fc4940689b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc498012500 tx=0x7fc49800a740 comp rx=0 tx=0).stop 2023-10-16T08:16:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80c9a80 msgr2=0x7fc4a80c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80c9a80 0x7fc4a80c75d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80c8e00 tx=0x7fc49c01b4e0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 shutdown_connections 2023-10-16T08:16:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80c9a80 0x7fc4a80c75d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4a8071e30 0x7fc4a80708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4940664f0 0x7fc4940689b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 --2- 172.21.15.32:0/4098970768 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc4a806c2e0 0x7fc4a81523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 >> 172.21.15.32:0/4098970768 conn(0x7fc4a80afe60 msgr2=0x7fc4a815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 shutdown_connections 2023-10-16T08:16:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.434+0000 7fc4ad53b640 1 -- 172.21.15.32:0/4098970768 wait complete. 2023-10-16T08:16:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.466+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8e94074010 con 0x7f8e940ada90 2023-10-16T08:16:52.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2023-10-16T08:16:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.466+0000 7f8e7bfff640 1 -- 172.21.15.32:0/1091473002 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e8401e090 con 0x7f8e940ada90 2023-10-16T08:16:52.474 INFO:teuthology.orchestra.run.smithi032.stdout:55834574859 2023-10-16T08:16:52.474 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1480005510 con 0x7f14980d3010 2023-10-16T08:16:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e80077580 msgr2=0x7f8e80079a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e80077580 0x7f8e80079a40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8e880af000 tx=0x7f8e880a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:52.476 INFO:teuthology.orchestra.run.smithi032.stdout:55834574858 2023-10-16T08:16:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f148affd640 1 -- 172.21.15.32:0/1577434255 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f148c0eed40 con 0x7f14980d3010 2023-10-16T08:16:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 msgr2=0x7f8e940c58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e940c58f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400b4a0 tx=0x7f8e8400b960 comp rx=0 tx=0).stop 2023-10-16T08:16:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 shutdown_connections 2023-10-16T08:16:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e940cb0e0 0x7f8e940c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e940c9900 0x7f8e940c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e80077580 0x7f8e80079a40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 --2- 172.21.15.32:0/1091473002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e940ada90 0x7f8e940c58f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 >> 172.21.15.32:0/1091473002 conn(0x7f8e9406c190 msgr2=0x7f8e940ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 shutdown_connections 2023-10-16T08:16:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.470+0000 7f8e9a0f7640 1 -- 172.21.15.32:0/1091473002 wait complete. 2023-10-16T08:16:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.474+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f147c066570 msgr2=0x7f147c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.474+0000 7f14a04ea640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f147c066570 0x7f147c068a30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1490007fd0 tx=0x7f1490009040 comp rx=0 tx=0).stop 2023-10-16T08:16:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.474+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 msgr2=0x7f149809c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.474+0000 7f14a04ea640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 0x7f149809c6c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f148c019030 tx=0x7f148c0191c0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 shutdown_connections 2023-10-16T08:16:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f147c066570 0x7f147c068a30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f149815cb90 0x7f14980b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14980d3010 0x7f149809c6c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 --2- 172.21.15.32:0/1577434255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14980d15d0 0x7f1498157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 >> 172.21.15.32:0/1577434255 conn(0x7f149809bc50 msgr2=0x7f14980d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 shutdown_connections 2023-10-16T08:16:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f14a04ea640 1 -- 172.21.15.32:0/1577434255 wait complete. 2023-10-16T08:16:52.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.478+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8a30005510 con 0x7f8a580c1e90 2023-10-16T08:16:52.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.486+0000 7f8a557fa640 1 -- 172.21.15.32:0/970056919 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a50054830 con 0x7f8a580c1e90 2023-10-16T08:16:52.490 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:52.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.486+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8a3c0664a0 msgr2=0x7f8a3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.486+0000 7f8a42ffd640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8a3c0664a0 0x7f8a3c068960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0c0040 tx=0x7f8a4c0046d0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.486+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 msgr2=0x7f8a580c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.486+0000 7f8a42ffd640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 0x7f8a580c2310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a580cb9d0 tx=0x7f8a5000fc70 comp rx=0 tx=0).stop 2023-10-16T08:16:52.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 shutdown_connections 2023-10-16T08:16:52.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a580c1e90 0x7f8a580c2310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a580c4540 0x7f8a580c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8a3c0664a0 0x7f8a3c068960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 --2- 172.21.15.32:0/970056919 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8a48003b50 0x7f8a580888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 >> 172.21.15.32:0/970056919 conn(0x7f8a580cad90 msgr2=0x7f8a580c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 shutdown_connections 2023-10-16T08:16:52.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7c580caeb0 con 0x7f7c580cba80 2023-10-16T08:16:52.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f8a42ffd640 1 -- 172.21.15.32:0/970056919 wait complete. 2023-10-16T08:16:52.497 INFO:teuthology.orchestra.run.smithi032.stdout:55834574859 2023-10-16T08:16:52.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.490+0000 7f7c557fa640 1 -- 172.21.15.32:0/3579764433 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c4c054b80 con 0x7f7c580cba80 2023-10-16T08:16:52.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7c3c0665a0 msgr2=0x7f7c3c068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7c3c0665a0 0x7f7c3c068a60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806b2a0 tx=0x7f7c44007c60 comp rx=0 tx=0).stop 2023-10-16T08:16:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 msgr2=0x7f7c580cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 0x7f7c580cbe60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00d830 tx=0x7f7c4c00dcf0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 shutdown_connections 2023-10-16T08:16:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c581f0210 0x7f7c581f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7c3c0665a0 0x7f7c3c068a60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7c5814d910 0x7f7c580b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 --2- 172.21.15.32:0/3579764433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c580cba80 0x7f7c580cbe60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 >> 172.21.15.32:0/3579764433 conn(0x7f7c5809bc50 msgr2=0x7f7c5806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 shutdown_connections 2023-10-16T08:16:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.494+0000 7f7c5ee46640 1 -- 172.21.15.32:0/3579764433 wait complete. 2023-10-16T08:16:52.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2023-10-16T08:16:52.503 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2023-10-16T08:16:52.519 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.518+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f77ac005510 con 0x7f77c8164520 2023-10-16T08:16:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.518+0000 7f77b6ffd640 1 -- 172.21.15.32:0/792275631 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77c0017030 con 0x7f77c8164520 2023-10-16T08:16:52.525 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77a8066500 msgr2=0x7f77a80689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77a8066500 0x7f77a80689c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77b801c470 tx=0x7f77b80a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8164520 msgr2=0x7f77c80c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8164520 0x7f77c80c6680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77c000caa0 tx=0x7f77c000cf60 comp rx=0 tx=0).stop 2023-10-16T08:16:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 shutdown_connections 2023-10-16T08:16:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8164520 0x7f77c80c6680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77c80c9b80 0x7f77c80c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77a8066500 0x7f77a80689c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 --2- 172.21.15.32:0/792275631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77c80ad960 0x7f77c814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.522+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 >> 172.21.15.32:0/792275631 conn(0x7f77c806c190 msgr2=0x7f77c80caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.526+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 shutdown_connections 2023-10-16T08:16:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.526+0000 7f77ccc54640 1 -- 172.21.15.32:0/792275631 wait complete. 2023-10-16T08:16:52.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2023-10-16T08:16:52.532 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2023-10-16T08:16:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.526+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feec4005510 con 0x7feee415cf00 2023-10-16T08:16:52.533 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.526+0000 7feee0ff9640 1 -- 172.21.15.32:0/2494750909 <== mon.2 v2:172.21.15.186:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feedc0ef5b0 con 0x7feee415cf00 2023-10-16T08:16:52.534 INFO:teuthology.orchestra.run.smithi032.stdout:55834574860 2023-10-16T08:16:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc0880d1850 con 0x7fc08815cb90 2023-10-16T08:16:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7fc076ffd640 1 -- 172.21.15.32:0/3683126825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc080016020 con 0x7fc08815cb90 2023-10-16T08:16:52.535 INFO:teuthology.orchestra.run.smithi032.stdout:55834574859 2023-10-16T08:16:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feed00664a0 msgr2=0x7feed0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feed00664a0 0x7feed0068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feed80011f0 tx=0x7feed8009040 comp rx=0 tx=0).stop 2023-10-16T08:16:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feee415cf00 msgr2=0x7feee4151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feee415cf00 0x7feee4151370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feedc0ab000 tx=0x7feedc0196e0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 shutdown_connections 2023-10-16T08:16:52.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feed00664a0 0x7feed0068960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feee415cf00 0x7feee4151370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee40d1410 0x7feee415b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 --2- 172.21.15.32:0/2494750909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feee40ce0b0 0x7feee40cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 >> 172.21.15.32:0/2494750909 conn(0x7feee40b4470 msgr2=0x7feee4157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 shutdown_connections 2023-10-16T08:16:52.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7feeca7fc640 1 -- 172.21.15.32:0/2494750909 wait complete. 2023-10-16T08:16:52.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0680664c0 msgr2=0x7fc068068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7fc08cd65640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0680664c0 0x7fc068068980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc07801aa30 tx=0x7fc0780a6000 comp rx=0 tx=0).stop 2023-10-16T08:16:52.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 msgr2=0x7fc0880b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.530+0000 7fc08cd65640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 0x7fc0880b5520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc08000cb90 tx=0x7fc08000a430 comp rx=0 tx=0).stop 2023-10-16T08:16:52.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 shutdown_connections 2023-10-16T08:16:52.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08815cb90 0x7fc0880b5520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0680664c0 0x7fc068068980 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0880d3010 0x7fc088201950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 --2- 172.21.15.32:0/3683126825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880d1400 0x7fc0880c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 >> 172.21.15.32:0/3683126825 conn(0x7fc08809bc50 msgr2=0x7fc08806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 shutdown_connections 2023-10-16T08:16:52.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.534+0000 7fc08cd65640 1 -- 172.21.15.32:0/3683126825 wait complete. 2023-10-16T08:16:52.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2023-10-16T08:16:52.560 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2023-10-16T08:16:52.563 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2023-10-16T08:16:52.567 DEBUG:teuthology.parallel:result is None 2023-10-16T08:16:52.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T08:16:52.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 Processor -- start 2023-10-16T08:16:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 -- start start 2023-10-16T08:16:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba6015ca50 0x7fba6006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fba600b21c0 0x7fba600b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba601607e0 con 0x7fba6015ce30 2023-10-16T08:16:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba600cb4a0 con 0x7fba6015ca50 2023-10-16T08:16:52.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba64809640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba600cf060 con 0x7fba600b21c0 2023-10-16T08:16:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba6015ca50 0x7fba6006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba6015ca50 0x7fba6006c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59432/0 (socket says 172.21.15.32:59432) 2023-10-16T08:16:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5e59c640 1 -- 172.21.15.32:0/1064266336 learned_addr learned my addr 172.21.15.32:0/1064266336 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5ed9d640 1 --2- 172.21.15.32:0/1064266336 >> v2:172.21.15.186:3300/0 conn(0x7fba600b21c0 0x7fba600b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5e59c640 1 -- 172.21.15.32:0/1064266336 >> v2:172.21.15.186:3300/0 conn(0x7fba600b21c0 msgr2=0x7fba600b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5e59c640 1 --2- 172.21.15.32:0/1064266336 >> v2:172.21.15.186:3300/0 conn(0x7fba600b21c0 0x7fba600b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5e59c640 1 -- 172.21.15.32:0/1064266336 >> v1:172.21.15.32:6789/0 conn(0x7fba6015ce30 legacy=0x7fba600cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.746+0000 7fba5dd9b640 1 -- 172.21.15.32:0/1064266336 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118955492 0 0) 0x7fba601607e0 con 0x7fba6015ce30 2023-10-16T08:16:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba5e59c640 1 -- 172.21.15.32:0/1064266336 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600b2ae0 con 0x7fba6015ca50 2023-10-16T08:16:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba5e59c640 1 --2- 172.21.15.32:0/1064266336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ca50 0x7fba6006c980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba600c4210 tx=0x7fba5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17d5e73d5951ab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba5dd9b640 1 -- 172.21.15.32:0/1064266336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500aa430 con 0x7fba6015ca50 2023-10-16T08:16:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba5dd9b640 1 -- 172.21.15.32:0/1064266336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba500aa590 con 0x7fba6015ca50 2023-10-16T08:16:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba5dd9b640 1 -- 172.21.15.32:0/1064266336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500aa920 con 0x7fba6015ca50 2023-10-16T08:16:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 -- 172.21.15.32:0/1064266336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ca50 msgr2=0x7fba6006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 --2- 172.21.15.32:0/1064266336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ca50 0x7fba6006c980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba600c4210 tx=0x7fba5001c4d0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 -- 172.21.15.32:0/1064266336 shutdown_connections 2023-10-16T08:16:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 --2- 172.21.15.32:0/1064266336 >> v2:172.21.15.186:3300/0 conn(0x7fba600b21c0 0x7fba600b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 --2- 172.21.15.32:0/1064266336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ca50 0x7fba6006c980 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 -- 172.21.15.32:0/1064266336 >> 172.21.15.32:0/1064266336 conn(0x7fba600cd1c0 msgr2=0x7fba600cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 -- 172.21.15.32:0/1064266336 shutdown_connections 2023-10-16T08:16:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.750+0000 7fba64809640 1 -- 172.21.15.32:0/1064266336 wait complete. 2023-10-16T08:16:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 Processor -- start 2023-10-16T08:16:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 -- start start 2023-10-16T08:16:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba600b21c0 0x7fba600b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba6015ca50 0x7fba601f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 0x7fba60153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60156db0 con 0x7fba6015ce30 2023-10-16T08:16:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba600c7940 con 0x7fba600b21c0 2023-10-16T08:16:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba601567f0 con 0x7fba6015ca50 2023-10-16T08:16:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 0x7fba60153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 0x7fba60153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59442/0 (socket says 172.21.15.32:59442) 2023-10-16T08:16:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 -- 172.21.15.32:0/388642060 learned_addr learned my addr 172.21.15.32:0/388642060 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5e59c640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba600b21c0 0x7fba600b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba57fff640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba6015ca50 0x7fba601f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 -- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba6015ca50 msgr2=0x7fba601f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba6015ca50 0x7fba601f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 -- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba600b21c0 msgr2=0x7fba600b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba600b21c0 0x7fba600b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 -- 172.21.15.32:0/388642060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500094d0 con 0x7fba6015ce30 2023-10-16T08:16:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba5ed9d640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 0x7fba60153fd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba58007e40 tx=0x7fba5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58014620 con 0x7fba6015ce30 2023-10-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5800ee90 con 0x7fba6015ce30 2023-10-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba580149e0 con 0x7fba6015ce30 2023-10-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba60156ad0 con 0x7fba6015ce30 2023-10-16T08:16:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.754+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601f4dd0 con 0x7fba6015ce30 2023-10-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.758+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba600c7b40 con 0x7fba6015ce30 2023-10-16T08:16:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.758+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba58014b40 con 0x7fba6015ce30 2023-10-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.758+0000 7fba56ffd640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba400664a0 0x7fba40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.758+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba58084bf0 con 0x7fba6015ce30 2023-10-16T08:16:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.758+0000 7fba5e59c640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba400664a0 0x7fba40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.762+0000 7fba5e59c640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba400664a0 0x7fba40068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba500ae000 tx=0x7fba500a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.762+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fba600c7b40 con 0x7fba6015ce30 2023-10-16T08:16:52.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.914+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba60070ab0 con 0x7fba400664a0 2023-10-16T08:16:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.926+0000 7fba56ffd640 1 -- 172.21.15.32:0/388642060 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345302 (secure 0 0 0) 0x7fba500aa960 con 0x7fba400664a0 2023-10-16T08:16:52.934 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:16:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba400664a0 msgr2=0x7fba40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba400664a0 0x7fba40068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba500ae000 tx=0x7fba500a7000 comp rx=0 tx=0).stop 2023-10-16T08:16:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 msgr2=0x7fba60153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 0x7fba60153fd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba58007e40 tx=0x7fba5800e4b0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 shutdown_connections 2023-10-16T08:16:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6015ce30 0x7fba60153fd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba400664a0 0x7fba40068960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fba6015ca50 0x7fba601f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 --2- 172.21.15.32:0/388642060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba600b21c0 0x7fba600b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 >> 172.21.15.32:0/388642060 conn(0x7fba600cd1c0 msgr2=0x7fba600afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 shutdown_connections 2023-10-16T08:16:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:52.930+0000 7fba64809640 1 -- 172.21.15.32:0/388642060 wait complete. 2023-10-16T08:16:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-16T08:16:52.021003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223628,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751092,"statfs":{"total":773094113280,"available":772865118208,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":225496635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003873"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988248+0000","last_change":"2023-10-16T08:16:21.956684+0000","last_active":"2023-10-16T08:16:26.988248+0000","last_peered":"2023-10-16T08:16:26.988248+0000","last_clean":"2023-10-16T08:16:26.988248+0000","last_became_active":"2023-10-16T08:16:21.949061+0000","last_became_peered":"2023-10-16T08:16:21.949061+0000","last_unstale":"2023-10-16T08:16:26.988248+0000","last_undegraded":"2023-10-16T08:16:26.988248+0000","last_fullsized":"2023-10-16T08:16:26.988248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985413+0000","last_change":"2023-10-16T08:16:22.972913+0000","last_active":"2023-10-16T08:16:26.985413+0000","last_peered":"2023-10-16T08:16:26.985413+0000","last_clean":"2023-10-16T08:16:26.985413+0000","last_became_active":"2023-10-16T08:16:22.972605+0000","last_became_peered":"2023-10-16T08:16:22.972605+0000","last_unstale":"2023-10-16T08:16:26.985413+0000","last_undegraded":"2023-10-16T08:16:26.985413+0000","last_fullsized":"2023-10-16T08:16:26.985413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989573+0000","last_change":"2023-10-16T08:16:27.182406+0000","last_active":"2023-10-16T08:16:27.989573+0000","last_peered":"2023-10-16T08:16:27.989573+0000","last_clean":"2023-10-16T08:16:27.989573+0000","last_became_active":"2023-10-16T08:16:27.182005+0000","last_became_peered":"2023-10-16T08:16:27.182005+0000","last_unstale":"2023-10-16T08:16:27.989573+0000","last_undegraded":"2023-10-16T08:16:27.989573+0000","last_fullsized":"2023-10-16T08:16:27.989573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:06.143173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985684+0000","last_change":"2023-10-16T08:16:21.945969+0000","last_active":"2023-10-16T08:16:26.985684+0000","last_peered":"2023-10-16T08:16:26.985684+0000","last_clean":"2023-10-16T08:16:26.985684+0000","last_became_active":"2023-10-16T08:16:21.945681+0000","last_became_peered":"2023-10-16T08:16:21.945681+0000","last_unstale":"2023-10-16T08:16:26.985684+0000","last_undegraded":"2023-10-16T08:16:26.985684+0000","last_fullsized":"2023-10-16T08:16:26.985684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988041+0000","last_change":"2023-10-16T08:16:22.974743+0000","last_active":"2023-10-16T08:16:26.988041+0000","last_peered":"2023-10-16T08:16:26.988041+0000","last_clean":"2023-10-16T08:16:26.988041+0000","last_became_active":"2023-10-16T08:16:22.974180+0000","last_became_peered":"2023-10-16T08:16:22.974180+0000","last_unstale":"2023-10-16T08:16:26.988041+0000","last_undegraded":"2023-10-16T08:16:26.988041+0000","last_fullsized":"2023-10-16T08:16:26.988041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542410+0000","last_change":"2023-10-16T08:16:27.011704+0000","last_active":"2023-10-16T08:16:29.542410+0000","last_peered":"2023-10-16T08:16:29.542410+0000","last_clean":"2023-10-16T08:16:29.542410+0000","last_became_active":"2023-10-16T08:16:27.011142+0000","last_became_peered":"2023-10-16T08:16:27.011142+0000","last_unstale":"2023-10-16T08:16:29.542410+0000","last_undegraded":"2023-10-16T08:16:29.542410+0000","last_fullsized":"2023-10-16T08:16:29.542410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:17.770482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178836+0000","last_change":"2023-10-16T08:16:21.964716+0000","last_active":"2023-10-16T08:16:27.178836+0000","last_peered":"2023-10-16T08:16:27.178836+0000","last_clean":"2023-10-16T08:16:27.178836+0000","last_became_active":"2023-10-16T08:16:21.964451+0000","last_became_peered":"2023-10-16T08:16:21.964451+0000","last_unstale":"2023-10-16T08:16:27.178836+0000","last_undegraded":"2023-10-16T08:16:27.178836+0000","last_fullsized":"2023-10-16T08:16:27.178836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994868+0000","last_change":"2023-10-16T08:16:22.971839+0000","last_active":"2023-10-16T08:16:26.994868+0000","last_peered":"2023-10-16T08:16:26.994868+0000","last_clean":"2023-10-16T08:16:26.994868+0000","last_became_active":"2023-10-16T08:16:22.971456+0000","last_became_peered":"2023-10-16T08:16:22.971456+0000","last_unstale":"2023-10-16T08:16:26.994868+0000","last_undegraded":"2023-10-16T08:16:26.994868+0000","last_fullsized":"2023-10-16T08:16:26.994868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543415+0000","last_change":"2023-10-16T08:16:27.024297+0000","last_active":"2023-10-16T08:16:29.543415+0000","last_peered":"2023-10-16T08:16:29.543415+0000","last_clean":"2023-10-16T08:16:29.543415+0000","last_became_active":"2023-10-16T08:16:27.023792+0000","last_became_peered":"2023-10-16T08:16:27.023792+0000","last_unstale":"2023-10-16T08:16:29.543415+0000","last_undegraded":"2023-10-16T08:16:29.543415+0000","last_fullsized":"2023-10-16T08:16:29.543415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:20.519605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178617+0000","last_change":"2023-10-16T08:16:21.964025+0000","last_active":"2023-10-16T08:16:27.178617+0000","last_peered":"2023-10-16T08:16:27.178617+0000","last_clean":"2023-10-16T08:16:27.178617+0000","last_became_active":"2023-10-16T08:16:21.963582+0000","last_became_peered":"2023-10-16T08:16:21.963582+0000","last_unstale":"2023-10-16T08:16:27.178617+0000","last_undegraded":"2023-10-16T08:16:27.178617+0000","last_fullsized":"2023-10-16T08:16:27.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994775+0000","last_change":"2023-10-16T08:16:22.980735+0000","last_active":"2023-10-16T08:16:26.994775+0000","last_peered":"2023-10-16T08:16:26.994775+0000","last_clean":"2023-10-16T08:16:26.994775+0000","last_became_active":"2023-10-16T08:16:22.980409+0000","last_became_peered":"2023-10-16T08:16:22.980409+0000","last_unstale":"2023-10-16T08:16:26.994775+0000","last_undegraded":"2023-10-16T08:16:26.994775+0000","last_fullsized":"2023-10-16T08:16:26.994775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542893+0000","last_change":"2023-10-16T08:16:27.024269+0000","last_active":"2023-10-16T08:16:29.542893+0000","last_peered":"2023-10-16T08:16:29.542893+0000","last_clean":"2023-10-16T08:16:29.542893+0000","last_became_active":"2023-10-16T08:16:27.023822+0000","last_became_peered":"2023-10-16T08:16:27.023822+0000","last_unstale":"2023-10-16T08:16:29.542893+0000","last_undegraded":"2023-10-16T08:16:29.542893+0000","last_fullsized":"2023-10-16T08:16:29.542893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:49.956062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987114+0000","last_change":"2023-10-16T08:16:22.969401+0000","last_active":"2023-10-16T08:16:26.987114+0000","last_peered":"2023-10-16T08:16:26.987114+0000","last_clean":"2023-10-16T08:16:26.987114+0000","last_became_active":"2023-10-16T08:16:22.969099+0000","last_became_peered":"2023-10-16T08:16:22.969099+0000","last_unstale":"2023-10-16T08:16:26.987114+0000","last_undegraded":"2023-10-16T08:16:26.987114+0000","last_fullsized":"2023-10-16T08:16:26.987114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178720+0000","last_change":"2023-10-16T08:16:21.961388+0000","last_active":"2023-10-16T08:16:27.178720+0000","last_peered":"2023-10-16T08:16:27.178720+0000","last_clean":"2023-10-16T08:16:27.178720+0000","last_became_active":"2023-10-16T08:16:21.961050+0000","last_became_peered":"2023-10-16T08:16:21.961050+0000","last_unstale":"2023-10-16T08:16:27.178720+0000","last_undegraded":"2023-10-16T08:16:27.178720+0000","last_fullsized":"2023-10-16T08:16:27.178720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:13.387501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543302+0000","last_change":"2023-10-16T08:16:27.019072+0000","last_active":"2023-10-16T08:16:29.543302+0000","last_peered":"2023-10-16T08:16:29.543302+0000","last_clean":"2023-10-16T08:16:29.543302+0000","last_became_active":"2023-10-16T08:16:27.018440+0000","last_became_peered":"2023-10-16T08:16:27.018440+0000","last_unstale":"2023-10-16T08:16:29.543302+0000","last_undegraded":"2023-10-16T08:16:29.543302+0000","last_fullsized":"2023-10-16T08:16:29.543302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:39.310251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990373+0000","last_change":"2023-10-16T08:16:22.960667+0000","last_active":"2023-10-16T08:16:26.990373+0000","last_peered":"2023-10-16T08:16:26.990373+0000","last_clean":"2023-10-16T08:16:26.990373+0000","last_became_active":"2023-10-16T08:16:22.960345+0000","last_became_peered":"2023-10-16T08:16:22.960345+0000","last_unstale":"2023-10-16T08:16:26.990373+0000","last_undegraded":"2023-10-16T08:16:26.990373+0000","last_fullsized":"2023-10-16T08:16:26.990373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013495+0000","last_change":"2023-10-16T08:16:21.950408+0000","last_active":"2023-10-16T08:16:27.013495+0000","last_peered":"2023-10-16T08:16:27.013495+0000","last_clean":"2023-10-16T08:16:27.013495+0000","last_became_active":"2023-10-16T08:16:21.947942+0000","last_became_peered":"2023-10-16T08:16:21.947942+0000","last_unstale":"2023-10-16T08:16:27.013495+0000","last_undegraded":"2023-10-16T08:16:27.013495+0000","last_fullsized":"2023-10-16T08:16:27.013495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542836+0000","last_change":"2023-10-16T08:16:27.021686+0000","last_active":"2023-10-16T08:16:29.542836+0000","last_peered":"2023-10-16T08:16:29.542836+0000","last_clean":"2023-10-16T08:16:29.542836+0000","last_became_active":"2023-10-16T08:16:27.021230+0000","last_became_peered":"2023-10-16T08:16:27.021230+0000","last_unstale":"2023-10-16T08:16:29.542836+0000","last_undegraded":"2023-10-16T08:16:29.542836+0000","last_fullsized":"2023-10-16T08:16:29.542836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:12.583792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986505+0000","last_change":"2023-10-16T08:16:22.955965+0000","last_active":"2023-10-16T08:16:26.986505+0000","last_peered":"2023-10-16T08:16:26.986505+0000","last_clean":"2023-10-16T08:16:26.986505+0000","last_became_active":"2023-10-16T08:16:22.955652+0000","last_became_peered":"2023-10-16T08:16:22.955652+0000","last_unstale":"2023-10-16T08:16:26.986505+0000","last_undegraded":"2023-10-16T08:16:26.986505+0000","last_fullsized":"2023-10-16T08:16:26.986505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009595+0000","last_change":"2023-10-16T08:16:21.950219+0000","last_active":"2023-10-16T08:16:27.009595+0000","last_peered":"2023-10-16T08:16:27.009595+0000","last_clean":"2023-10-16T08:16:27.009595+0000","last_became_active":"2023-10-16T08:16:21.949749+0000","last_became_peered":"2023-10-16T08:16:21.949749+0000","last_unstale":"2023-10-16T08:16:27.009595+0000","last_undegraded":"2023-10-16T08:16:27.009595+0000","last_fullsized":"2023-10-16T08:16:27.009595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988141+0000","last_change":"2023-10-16T08:16:27.182341+0000","last_active":"2023-10-16T08:16:27.988141+0000","last_peered":"2023-10-16T08:16:27.988141+0000","last_clean":"2023-10-16T08:16:27.988141+0000","last_became_active":"2023-10-16T08:16:27.181854+0000","last_became_peered":"2023-10-16T08:16:27.181854+0000","last_unstale":"2023-10-16T08:16:27.988141+0000","last_undegraded":"2023-10-16T08:16:27.988141+0000","last_fullsized":"2023-10-16T08:16:27.988141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003825+0000","last_change":"2023-10-16T08:16:22.962471+0000","last_active":"2023-10-16T08:16:27.003825+0000","last_peered":"2023-10-16T08:16:27.003825+0000","last_clean":"2023-10-16T08:16:27.003825+0000","last_became_active":"2023-10-16T08:16:22.961915+0000","last_became_peered":"2023-10-16T08:16:22.961915+0000","last_unstale":"2023-10-16T08:16:27.003825+0000","last_undegraded":"2023-10-16T08:16:27.003825+0000","last_fullsized":"2023-10-16T08:16:27.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:13.023207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177873+0000","last_change":"2023-10-16T08:16:21.964429+0000","last_active":"2023-10-16T08:16:27.177873+0000","last_peered":"2023-10-16T08:16:27.177873+0000","last_clean":"2023-10-16T08:16:27.177873+0000","last_became_active":"2023-10-16T08:16:21.964154+0000","last_became_peered":"2023-10-16T08:16:21.964154+0000","last_unstale":"2023-10-16T08:16:27.177873+0000","last_undegraded":"2023-10-16T08:16:27.177873+0000","last_fullsized":"2023-10-16T08:16:27.177873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543629+0000","last_change":"2023-10-16T08:16:27.027359+0000","last_active":"2023-10-16T08:16:29.543629+0000","last_peered":"2023-10-16T08:16:29.543629+0000","last_clean":"2023-10-16T08:16:29.543629+0000","last_became_active":"2023-10-16T08:16:27.026923+0000","last_became_peered":"2023-10-16T08:16:27.026923+0000","last_unstale":"2023-10-16T08:16:29.543629+0000","last_undegraded":"2023-10-16T08:16:29.543629+0000","last_fullsized":"2023-10-16T08:16:29.543629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.989245+0000","last_change":"2023-10-16T08:16:22.967177+0000","last_active":"2023-10-16T08:16:26.989245+0000","last_peered":"2023-10-16T08:16:26.989245+0000","last_clean":"2023-10-16T08:16:26.989245+0000","last_became_active":"2023-10-16T08:16:22.966812+0000","last_became_peered":"2023-10-16T08:16:22.966812+0000","last_unstale":"2023-10-16T08:16:26.989245+0000","last_undegraded":"2023-10-16T08:16:26.989245+0000","last_fullsized":"2023-10-16T08:16:26.989245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988555+0000","last_change":"2023-10-16T08:16:21.958338+0000","last_active":"2023-10-16T08:16:26.988555+0000","last_peered":"2023-10-16T08:16:26.988555+0000","last_clean":"2023-10-16T08:16:26.988555+0000","last_became_active":"2023-10-16T08:16:21.958052+0000","last_became_peered":"2023-10-16T08:16:21.958052+0000","last_unstale":"2023-10-16T08:16:26.988555+0000","last_undegraded":"2023-10-16T08:16:26.988555+0000","last_fullsized":"2023-10-16T08:16:26.988555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543530+0000","last_change":"2023-10-16T08:16:27.184138+0000","last_active":"2023-10-16T08:16:29.543530+0000","last_peered":"2023-10-16T08:16:29.543530+0000","last_clean":"2023-10-16T08:16:29.543530+0000","last_became_active":"2023-10-16T08:16:27.183769+0000","last_became_peered":"2023-10-16T08:16:27.183769+0000","last_unstale":"2023-10-16T08:16:29.543530+0000","last_undegraded":"2023-10-16T08:16:29.543530+0000","last_fullsized":"2023-10-16T08:16:29.543530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987117+0000","last_change":"2023-10-16T08:16:22.973708+0000","last_active":"2023-10-16T08:16:26.987117+0000","last_peered":"2023-10-16T08:16:26.987117+0000","last_clean":"2023-10-16T08:16:26.987117+0000","last_became_active":"2023-10-16T08:16:22.973402+0000","last_became_peered":"2023-10-16T08:16:22.973402+0000","last_unstale":"2023-10-16T08:16:26.987117+0000","last_undegraded":"2023-10-16T08:16:26.987117+0000","last_fullsized":"2023-10-16T08:16:26.987117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014342+0000","last_change":"2023-10-16T08:16:21.955020+0000","last_active":"2023-10-16T08:16:27.014342+0000","last_peered":"2023-10-16T08:16:27.014342+0000","last_clean":"2023-10-16T08:16:27.014342+0000","last_became_active":"2023-10-16T08:16:21.954681+0000","last_became_peered":"2023-10-16T08:16:21.954681+0000","last_unstale":"2023-10-16T08:16:27.014342+0000","last_undegraded":"2023-10-16T08:16:27.014342+0000","last_fullsized":"2023-10-16T08:16:27.014342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989486+0000","last_change":"2023-10-16T08:16:27.179578+0000","last_active":"2023-10-16T08:16:27.989486+0000","last_peered":"2023-10-16T08:16:27.989486+0000","last_clean":"2023-10-16T08:16:27.989486+0000","last_became_active":"2023-10-16T08:16:27.178683+0000","last_became_peered":"2023-10-16T08:16:27.178683+0000","last_unstale":"2023-10-16T08:16:27.989486+0000","last_undegraded":"2023-10-16T08:16:27.989486+0000","last_fullsized":"2023-10-16T08:16:27.989486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014216+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.014216+0000","last_peered":"2023-10-16T08:16:27.014216+0000","last_clean":"2023-10-16T08:16:27.014216+0000","last_became_active":"2023-10-16T08:16:22.952824+0000","last_became_peered":"2023-10-16T08:16:22.952824+0000","last_unstale":"2023-10-16T08:16:27.014216+0000","last_undegraded":"2023-10-16T08:16:27.014216+0000","last_fullsized":"2023-10-16T08:16:27.014216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178889+0000","last_change":"2023-10-16T08:16:21.964665+0000","last_active":"2023-10-16T08:16:27.178889+0000","last_peered":"2023-10-16T08:16:27.178889+0000","last_clean":"2023-10-16T08:16:27.178889+0000","last_became_active":"2023-10-16T08:16:21.964105+0000","last_became_peered":"2023-10-16T08:16:21.964105+0000","last_unstale":"2023-10-16T08:16:27.178889+0000","last_undegraded":"2023-10-16T08:16:27.178889+0000","last_fullsized":"2023-10-16T08:16:27.178889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:12.709836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541103+0000","last_change":"2023-10-16T08:16:27.180634+0000","last_active":"2023-10-16T08:16:29.541103+0000","last_peered":"2023-10-16T08:16:29.541103+0000","last_clean":"2023-10-16T08:16:29.541103+0000","last_became_active":"2023-10-16T08:16:27.180186+0000","last_became_peered":"2023-10-16T08:16:27.180186+0000","last_unstale":"2023-10-16T08:16:29.541103+0000","last_undegraded":"2023-10-16T08:16:29.541103+0000","last_fullsized":"2023-10-16T08:16:29.541103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994880+0000","last_change":"2023-10-16T08:16:22.994702+0000","last_active":"2023-10-16T08:16:26.994880+0000","last_peered":"2023-10-16T08:16:26.994880+0000","last_clean":"2023-10-16T08:16:26.994880+0000","last_became_active":"2023-10-16T08:16:22.990441+0000","last_became_peered":"2023-10-16T08:16:22.990441+0000","last_unstale":"2023-10-16T08:16:26.994880+0000","last_undegraded":"2023-10-16T08:16:26.994880+0000","last_fullsized":"2023-10-16T08:16:26.994880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014301+0000","last_change":"2023-10-16T08:16:21.942771+0000","last_active":"2023-10-16T08:16:27.014301+0000","last_peered":"2023-10-16T08:16:27.014301+0000","last_clean":"2023-10-16T08:16:27.014301+0000","last_became_active":"2023-10-16T08:16:21.942441+0000","last_became_peered":"2023-10-16T08:16:21.942441+0000","last_unstale":"2023-10-16T08:16:27.014301+0000","last_undegraded":"2023-10-16T08:16:27.014301+0000","last_fullsized":"2023-10-16T08:16:27.014301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.986815+0000","last_change":"2023-10-16T08:16:27.013697+0000","last_active":"2023-10-16T08:16:27.986815+0000","last_peered":"2023-10-16T08:16:27.986815+0000","last_clean":"2023-10-16T08:16:27.986815+0000","last_became_active":"2023-10-16T08:16:27.012937+0000","last_became_peered":"2023-10-16T08:16:27.012937+0000","last_unstale":"2023-10-16T08:16:27.986815+0000","last_undegraded":"2023-10-16T08:16:27.986815+0000","last_fullsized":"2023-10-16T08:16:27.986815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003417+0000","last_change":"2023-10-16T08:16:22.964069+0000","last_active":"2023-10-16T08:16:27.003417+0000","last_peered":"2023-10-16T08:16:27.003417+0000","last_clean":"2023-10-16T08:16:27.003417+0000","last_became_active":"2023-10-16T08:16:22.963802+0000","last_became_peered":"2023-10-16T08:16:22.963802+0000","last_unstale":"2023-10-16T08:16:27.003417+0000","last_undegraded":"2023-10-16T08:16:27.003417+0000","last_fullsized":"2023-10-16T08:16:27.003417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994257+0000","last_change":"2023-10-16T08:16:21.962674+0000","last_active":"2023-10-16T08:16:26.994257+0000","last_peered":"2023-10-16T08:16:26.994257+0000","last_clean":"2023-10-16T08:16:26.994257+0000","last_became_active":"2023-10-16T08:16:21.962316+0000","last_became_peered":"2023-10-16T08:16:21.962316+0000","last_unstale":"2023-10-16T08:16:26.994257+0000","last_undegraded":"2023-10-16T08:16:26.994257+0000","last_fullsized":"2023-10-16T08:16:26.994257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:32.032856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987533+0000","last_change":"2023-10-16T08:16:27.020747+0000","last_active":"2023-10-16T08:16:27.987533+0000","last_peered":"2023-10-16T08:16:27.987533+0000","last_clean":"2023-10-16T08:16:27.987533+0000","last_became_active":"2023-10-16T08:16:27.019634+0000","last_became_peered":"2023-10-16T08:16:27.019634+0000","last_unstale":"2023-10-16T08:16:27.987533+0000","last_undegraded":"2023-10-16T08:16:27.987533+0000","last_fullsized":"2023-10-16T08:16:27.987533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009729+0000","last_change":"2023-10-16T08:16:22.953894+0000","last_active":"2023-10-16T08:16:27.009729+0000","last_peered":"2023-10-16T08:16:27.009729+0000","last_clean":"2023-10-16T08:16:27.009729+0000","last_became_active":"2023-10-16T08:16:22.953606+0000","last_became_peered":"2023-10-16T08:16:22.953606+0000","last_unstale":"2023-10-16T08:16:27.009729+0000","last_undegraded":"2023-10-16T08:16:27.009729+0000","last_fullsized":"2023-10-16T08:16:27.009729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996654+0000","last_change":"2023-10-16T08:16:21.946996+0000","last_active":"2023-10-16T08:16:26.996654+0000","last_peered":"2023-10-16T08:16:26.996654+0000","last_clean":"2023-10-16T08:16:26.996654+0000","last_became_active":"2023-10-16T08:16:21.946675+0000","last_became_peered":"2023-10-16T08:16:21.946675+0000","last_unstale":"2023-10-16T08:16:26.996654+0000","last_undegraded":"2023-10-16T08:16:26.996654+0000","last_fullsized":"2023-10-16T08:16:26.996654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:43:55.303584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988192+0000","last_change":"2023-10-16T08:16:27.020112+0000","last_active":"2023-10-16T08:16:27.988192+0000","last_peered":"2023-10-16T08:16:27.988192+0000","last_clean":"2023-10-16T08:16:27.988192+0000","last_became_active":"2023-10-16T08:16:27.019606+0000","last_became_peered":"2023-10-16T08:16:27.019606+0000","last_unstale":"2023-10-16T08:16:27.988192+0000","last_undegraded":"2023-10-16T08:16:27.988192+0000","last_fullsized":"2023-10-16T08:16:27.988192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984195+0000","last_change":"2023-10-16T08:16:22.962655+0000","last_active":"2023-10-16T08:16:26.984195+0000","last_peered":"2023-10-16T08:16:26.984195+0000","last_clean":"2023-10-16T08:16:26.984195+0000","last_became_active":"2023-10-16T08:16:22.962344+0000","last_became_peered":"2023-10-16T08:16:22.962344+0000","last_unstale":"2023-10-16T08:16:26.984195+0000","last_undegraded":"2023-10-16T08:16:26.984195+0000","last_fullsized":"2023-10-16T08:16:26.984195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:43.247033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177937+0000","last_change":"2023-10-16T08:16:21.959357+0000","last_active":"2023-10-16T08:16:27.177937+0000","last_peered":"2023-10-16T08:16:27.177937+0000","last_clean":"2023-10-16T08:16:27.177937+0000","last_became_active":"2023-10-16T08:16:21.959015+0000","last_became_peered":"2023-10-16T08:16:21.959015+0000","last_unstale":"2023-10-16T08:16:27.177937+0000","last_undegraded":"2023-10-16T08:16:27.177937+0000","last_fullsized":"2023-10-16T08:16:27.177937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:02.223178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541384+0000","last_change":"2023-10-16T08:16:27.179195+0000","last_active":"2023-10-16T08:16:29.541384+0000","last_peered":"2023-10-16T08:16:29.541384+0000","last_clean":"2023-10-16T08:16:29.541384+0000","last_became_active":"2023-10-16T08:16:27.178732+0000","last_became_peered":"2023-10-16T08:16:27.178732+0000","last_unstale":"2023-10-16T08:16:29.541384+0000","last_undegraded":"2023-10-16T08:16:29.541384+0000","last_fullsized":"2023-10-16T08:16:29.541384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986752+0000","last_change":"2023-10-16T08:16:22.973605+0000","last_active":"2023-10-16T08:16:26.986752+0000","last_peered":"2023-10-16T08:16:26.986752+0000","last_clean":"2023-10-16T08:16:26.986752+0000","last_became_active":"2023-10-16T08:16:22.973281+0000","last_became_peered":"2023-10-16T08:16:22.973281+0000","last_unstale":"2023-10-16T08:16:26.986752+0000","last_undegraded":"2023-10-16T08:16:26.986752+0000","last_fullsized":"2023-10-16T08:16:26.986752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994407+0000","last_change":"2023-10-16T08:16:21.950850+0000","last_active":"2023-10-16T08:16:26.994407+0000","last_peered":"2023-10-16T08:16:26.994407+0000","last_clean":"2023-10-16T08:16:26.994407+0000","last_became_active":"2023-10-16T08:16:21.950526+0000","last_became_peered":"2023-10-16T08:16:21.950526+0000","last_unstale":"2023-10-16T08:16:26.994407+0000","last_undegraded":"2023-10-16T08:16:26.994407+0000","last_fullsized":"2023-10-16T08:16:26.994407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177966+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:16:27.177966+0000","last_peered":"2023-10-16T08:16:27.177966+0000","last_clean":"2023-10-16T08:16:27.177966+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:16:27.177966+0000","last_undegraded":"2023-10-16T08:16:27.177966+0000","last_fullsized":"2023-10-16T08:16:27.177966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991969+0000","last_change":"2023-10-16T08:16:27.184250+0000","last_active":"2023-10-16T08:16:27.991969+0000","last_peered":"2023-10-16T08:16:27.991969+0000","last_clean":"2023-10-16T08:16:27.991969+0000","last_became_active":"2023-10-16T08:16:27.183826+0000","last_became_peered":"2023-10-16T08:16:27.183826+0000","last_unstale":"2023-10-16T08:16:27.991969+0000","last_undegraded":"2023-10-16T08:16:27.991969+0000","last_fullsized":"2023-10-16T08:16:27.991969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178713+0000","last_change":"2023-10-16T08:16:22.975303+0000","last_active":"2023-10-16T08:16:27.178713+0000","last_peered":"2023-10-16T08:16:27.178713+0000","last_clean":"2023-10-16T08:16:27.178713+0000","last_became_active":"2023-10-16T08:16:22.974890+0000","last_became_peered":"2023-10-16T08:16:22.974890+0000","last_unstale":"2023-10-16T08:16:27.178713+0000","last_undegraded":"2023-10-16T08:16:27.178713+0000","last_fullsized":"2023-10-16T08:16:27.178713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.997283+0000","last_change":"2023-10-16T08:16:21.959337+0000","last_active":"2023-10-16T08:16:26.997283+0000","last_peered":"2023-10-16T08:16:26.997283+0000","last_clean":"2023-10-16T08:16:26.997283+0000","last_became_active":"2023-10-16T08:16:21.959059+0000","last_became_peered":"2023-10-16T08:16:21.959059+0000","last_unstale":"2023-10-16T08:16:26.997283+0000","last_undegraded":"2023-10-16T08:16:26.997283+0000","last_fullsized":"2023-10-16T08:16:26.997283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991525+0000","last_change":"2023-10-16T08:16:27.181051+0000","last_active":"2023-10-16T08:16:27.991525+0000","last_peered":"2023-10-16T08:16:27.991525+0000","last_clean":"2023-10-16T08:16:27.991525+0000","last_became_active":"2023-10-16T08:16:27.180503+0000","last_became_peered":"2023-10-16T08:16:27.180503+0000","last_unstale":"2023-10-16T08:16:27.991525+0000","last_undegraded":"2023-10-16T08:16:27.991525+0000","last_fullsized":"2023-10-16T08:16:27.991525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994965+0000","last_change":"2023-10-16T08:16:22.990070+0000","last_active":"2023-10-16T08:16:26.994965+0000","last_peered":"2023-10-16T08:16:26.994965+0000","last_clean":"2023-10-16T08:16:26.994965+0000","last_became_active":"2023-10-16T08:16:22.989787+0000","last_became_peered":"2023-10-16T08:16:22.989787+0000","last_unstale":"2023-10-16T08:16:26.994965+0000","last_undegraded":"2023-10-16T08:16:26.994965+0000","last_fullsized":"2023-10-16T08:16:26.994965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013982+0000","last_change":"2023-10-16T08:16:21.961287+0000","last_active":"2023-10-16T08:16:27.013982+0000","last_peered":"2023-10-16T08:16:27.013982+0000","last_clean":"2023-10-16T08:16:27.013982+0000","last_became_active":"2023-10-16T08:16:21.956293+0000","last_became_peered":"2023-10-16T08:16:21.956293+0000","last_unstale":"2023-10-16T08:16:27.013982+0000","last_undegraded":"2023-10-16T08:16:27.013982+0000","last_fullsized":"2023-10-16T08:16:27.013982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541257+0000","last_change":"2023-10-16T08:16:27.013097+0000","last_active":"2023-10-16T08:16:29.541257+0000","last_peered":"2023-10-16T08:16:29.541257+0000","last_clean":"2023-10-16T08:16:29.541257+0000","last_became_active":"2023-10-16T08:16:27.012587+0000","last_became_peered":"2023-10-16T08:16:27.012587+0000","last_unstale":"2023-10-16T08:16:29.541257+0000","last_undegraded":"2023-10-16T08:16:29.541257+0000","last_fullsized":"2023-10-16T08:16:29.541257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179067+0000","last_change":"2023-10-16T08:16:22.979584+0000","last_active":"2023-10-16T08:16:27.179067+0000","last_peered":"2023-10-16T08:16:27.179067+0000","last_clean":"2023-10-16T08:16:27.179067+0000","last_became_active":"2023-10-16T08:16:22.978239+0000","last_became_peered":"2023-10-16T08:16:22.978239+0000","last_unstale":"2023-10-16T08:16:27.179067+0000","last_undegraded":"2023-10-16T08:16:27.179067+0000","last_fullsized":"2023-10-16T08:16:27.179067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:27:57.200006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996247+0000","last_change":"2023-10-16T08:16:21.947037+0000","last_active":"2023-10-16T08:16:26.996247+0000","last_peered":"2023-10-16T08:16:26.996247+0000","last_clean":"2023-10-16T08:16:26.996247+0000","last_became_active":"2023-10-16T08:16:21.946671+0000","last_became_peered":"2023-10-16T08:16:21.946671+0000","last_unstale":"2023-10-16T08:16:26.996247+0000","last_undegraded":"2023-10-16T08:16:26.996247+0000","last_fullsized":"2023-10-16T08:16:26.996247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542197+0000","last_change":"2023-10-16T08:16:27.189477+0000","last_active":"2023-10-16T08:16:29.542197+0000","last_peered":"2023-10-16T08:16:29.542197+0000","last_clean":"2023-10-16T08:16:29.542197+0000","last_became_active":"2023-10-16T08:16:27.185942+0000","last_became_peered":"2023-10-16T08:16:27.185942+0000","last_unstale":"2023-10-16T08:16:29.542197+0000","last_undegraded":"2023-10-16T08:16:29.542197+0000","last_fullsized":"2023-10-16T08:16:29.542197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:13.645047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.973600+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.973215+0000","last_became_peered":"2023-10-16T08:16:22.973215+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.005005+0000","last_change":"2023-10-16T08:16:21.949416+0000","last_active":"2023-10-16T08:16:27.005005+0000","last_peered":"2023-10-16T08:16:27.005005+0000","last_clean":"2023-10-16T08:16:27.005005+0000","last_became_active":"2023-10-16T08:16:21.948836+0000","last_became_peered":"2023-10-16T08:16:21.948836+0000","last_unstale":"2023-10-16T08:16:27.005005+0000","last_undegraded":"2023-10-16T08:16:27.005005+0000","last_fullsized":"2023-10-16T08:16:27.005005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543245+0000","last_change":"2023-10-16T08:16:27.012879+0000","last_active":"2023-10-16T08:16:29.543245+0000","last_peered":"2023-10-16T08:16:29.543245+0000","last_clean":"2023-10-16T08:16:29.543245+0000","last_became_active":"2023-10-16T08:16:27.012466+0000","last_became_peered":"2023-10-16T08:16:27.012466+0000","last_unstale":"2023-10-16T08:16:29.543245+0000","last_undegraded":"2023-10-16T08:16:29.543245+0000","last_fullsized":"2023-10-16T08:16:29.543245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:16.486175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178093+0000","last_change":"2023-10-16T08:16:22.973193+0000","last_active":"2023-10-16T08:16:27.178093+0000","last_peered":"2023-10-16T08:16:27.178093+0000","last_clean":"2023-10-16T08:16:27.178093+0000","last_became_active":"2023-10-16T08:16:22.972472+0000","last_became_peered":"2023-10-16T08:16:22.972472+0000","last_unstale":"2023-10-16T08:16:27.178093+0000","last_undegraded":"2023-10-16T08:16:27.178093+0000","last_fullsized":"2023-10-16T08:16:27.178093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996389+0000","last_change":"2023-10-16T08:16:21.952668+0000","last_active":"2023-10-16T08:16:26.996389+0000","last_peered":"2023-10-16T08:16:26.996389+0000","last_clean":"2023-10-16T08:16:26.996389+0000","last_became_active":"2023-10-16T08:16:21.952006+0000","last_became_peered":"2023-10-16T08:16:21.952006+0000","last_unstale":"2023-10-16T08:16:26.996389+0000","last_undegraded":"2023-10-16T08:16:26.996389+0000","last_fullsized":"2023-10-16T08:16:26.996389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541359+0000","last_change":"2023-10-16T08:16:27.015675+0000","last_active":"2023-10-16T08:16:29.541359+0000","last_peered":"2023-10-16T08:16:29.541359+0000","last_clean":"2023-10-16T08:16:29.541359+0000","last_became_active":"2023-10-16T08:16:27.015279+0000","last_became_peered":"2023-10-16T08:16:27.015279+0000","last_unstale":"2023-10-16T08:16:29.541359+0000","last_undegraded":"2023-10-16T08:16:29.541359+0000","last_fullsized":"2023-10-16T08:16:29.541359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986776+0000","last_change":"2023-10-16T08:16:22.963194+0000","last_active":"2023-10-16T08:16:26.986776+0000","last_peered":"2023-10-16T08:16:26.986776+0000","last_clean":"2023-10-16T08:16:26.986776+0000","last_became_active":"2023-10-16T08:16:22.962589+0000","last_became_peered":"2023-10-16T08:16:22.962589+0000","last_unstale":"2023-10-16T08:16:26.986776+0000","last_undegraded":"2023-10-16T08:16:26.986776+0000","last_fullsized":"2023-10-16T08:16:26.986776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986872+0000","last_change":"2023-10-16T08:16:21.957192+0000","last_active":"2023-10-16T08:16:26.986872+0000","last_peered":"2023-10-16T08:16:26.986872+0000","last_clean":"2023-10-16T08:16:26.986872+0000","last_became_active":"2023-10-16T08:16:21.955564+0000","last_became_peered":"2023-10-16T08:16:21.955564+0000","last_unstale":"2023-10-16T08:16:26.986872+0000","last_undegraded":"2023-10-16T08:16:26.986872+0000","last_fullsized":"2023-10-16T08:16:26.986872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:58.308103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990800+0000","last_change":"2023-10-16T08:16:27.185040+0000","last_active":"2023-10-16T08:16:27.990800+0000","last_peered":"2023-10-16T08:16:27.990800+0000","last_clean":"2023-10-16T08:16:27.990800+0000","last_became_active":"2023-10-16T08:16:27.184448+0000","last_became_peered":"2023-10-16T08:16:27.184448+0000","last_unstale":"2023-10-16T08:16:27.990800+0000","last_undegraded":"2023-10-16T08:16:27.990800+0000","last_fullsized":"2023-10-16T08:16:27.990800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:59.852030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178532+0000","last_change":"2023-10-16T08:16:22.980031+0000","last_active":"2023-10-16T08:16:27.178532+0000","last_peered":"2023-10-16T08:16:27.178532+0000","last_clean":"2023-10-16T08:16:27.178532+0000","last_became_active":"2023-10-16T08:16:22.976708+0000","last_became_peered":"2023-10-16T08:16:22.976708+0000","last_unstale":"2023-10-16T08:16:27.178532+0000","last_undegraded":"2023-10-16T08:16:27.178532+0000","last_fullsized":"2023-10-16T08:16:27.178532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994242+0000","last_change":"2023-10-16T08:16:21.956986+0000","last_active":"2023-10-16T08:16:26.994242+0000","last_peered":"2023-10-16T08:16:26.994242+0000","last_clean":"2023-10-16T08:16:26.994242+0000","last_became_active":"2023-10-16T08:16:21.956307+0000","last_became_peered":"2023-10-16T08:16:21.956307+0000","last_unstale":"2023-10-16T08:16:26.994242+0000","last_undegraded":"2023-10-16T08:16:26.994242+0000","last_fullsized":"2023-10-16T08:16:26.994242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:26.142575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990801+0000","last_change":"2023-10-16T08:16:27.179150+0000","last_active":"2023-10-16T08:16:27.990801+0000","last_peered":"2023-10-16T08:16:27.990801+0000","last_clean":"2023-10-16T08:16:27.990801+0000","last_became_active":"2023-10-16T08:16:27.178649+0000","last_became_peered":"2023-10-16T08:16:27.178649+0000","last_unstale":"2023-10-16T08:16:27.990801+0000","last_undegraded":"2023-10-16T08:16:27.990801+0000","last_fullsized":"2023-10-16T08:16:27.990801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:45.567047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984308+0000","last_change":"2023-10-16T08:16:22.967505+0000","last_active":"2023-10-16T08:16:26.984308+0000","last_peered":"2023-10-16T08:16:26.984308+0000","last_clean":"2023-10-16T08:16:26.984308+0000","last_became_active":"2023-10-16T08:16:22.967214+0000","last_became_peered":"2023-10-16T08:16:22.967214+0000","last_unstale":"2023-10-16T08:16:26.984308+0000","last_undegraded":"2023-10-16T08:16:26.984308+0000","last_fullsized":"2023-10-16T08:16:26.984308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988187+0000","last_change":"2023-10-16T08:16:21.946248+0000","last_active":"2023-10-16T08:16:26.988187+0000","last_peered":"2023-10-16T08:16:26.988187+0000","last_clean":"2023-10-16T08:16:26.988187+0000","last_became_active":"2023-10-16T08:16:21.945958+0000","last_became_peered":"2023-10-16T08:16:21.945958+0000","last_unstale":"2023-10-16T08:16:26.988187+0000","last_undegraded":"2023-10-16T08:16:26.988187+0000","last_fullsized":"2023-10-16T08:16:26.988187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543632+0000","last_change":"2023-10-16T08:16:27.184504+0000","last_active":"2023-10-16T08:16:29.543632+0000","last_peered":"2023-10-16T08:16:29.543632+0000","last_clean":"2023-10-16T08:16:29.543632+0000","last_became_active":"2023-10-16T08:16:27.184029+0000","last_became_peered":"2023-10-16T08:16:27.184029+0000","last_unstale":"2023-10-16T08:16:29.543632+0000","last_undegraded":"2023-10-16T08:16:29.543632+0000","last_fullsized":"2023-10-16T08:16:29.543632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178733+0000","last_change":"2023-10-16T08:16:22.986066+0000","last_active":"2023-10-16T08:16:27.178733+0000","last_peered":"2023-10-16T08:16:27.178733+0000","last_clean":"2023-10-16T08:16:27.178733+0000","last_became_active":"2023-10-16T08:16:22.985791+0000","last_became_peered":"2023-10-16T08:16:22.985791+0000","last_unstale":"2023-10-16T08:16:27.178733+0000","last_undegraded":"2023-10-16T08:16:27.178733+0000","last_fullsized":"2023-10-16T08:16:27.178733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:38.000149+0000","last_change":"2023-10-16T08:16:21.955781+0000","last_active":"2023-10-16T08:16:38.000149+0000","last_peered":"2023-10-16T08:16:38.000149+0000","last_clean":"2023-10-16T08:16:38.000149+0000","last_became_active":"2023-10-16T08:16:21.955327+0000","last_became_peered":"2023-10-16T08:16:21.955327+0000","last_unstale":"2023-10-16T08:16:38.000149+0000","last_undegraded":"2023-10-16T08:16:38.000149+0000","last_fullsized":"2023-10-16T08:16:38.000149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987157+0000","last_change":"2023-10-16T08:16:27.007137+0000","last_active":"2023-10-16T08:16:27.987157+0000","last_peered":"2023-10-16T08:16:27.987157+0000","last_clean":"2023-10-16T08:16:27.987157+0000","last_became_active":"2023-10-16T08:16:27.006722+0000","last_became_peered":"2023-10-16T08:16:27.006722+0000","last_unstale":"2023-10-16T08:16:27.987157+0000","last_undegraded":"2023-10-16T08:16:27.987157+0000","last_fullsized":"2023-10-16T08:16:27.987157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004799+0000","last_change":"2023-10-16T08:16:22.976159+0000","last_active":"2023-10-16T08:16:27.004799+0000","last_peered":"2023-10-16T08:16:27.004799+0000","last_clean":"2023-10-16T08:16:27.004799+0000","last_became_active":"2023-10-16T08:16:22.975879+0000","last_became_peered":"2023-10-16T08:16:22.975879+0000","last_unstale":"2023-10-16T08:16:27.004799+0000","last_undegraded":"2023-10-16T08:16:27.004799+0000","last_fullsized":"2023-10-16T08:16:27.004799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:04.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013901+0000","last_change":"2023-10-16T08:16:21.961500+0000","last_active":"2023-10-16T08:16:27.013901+0000","last_peered":"2023-10-16T08:16:27.013901+0000","last_clean":"2023-10-16T08:16:27.013901+0000","last_became_active":"2023-10-16T08:16:21.959870+0000","last_became_peered":"2023-10-16T08:16:21.959870+0000","last_unstale":"2023-10-16T08:16:27.013901+0000","last_undegraded":"2023-10-16T08:16:27.013901+0000","last_fullsized":"2023-10-16T08:16:27.013901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543647+0000","last_change":"2023-10-16T08:16:27.018940+0000","last_active":"2023-10-16T08:16:29.543647+0000","last_peered":"2023-10-16T08:16:29.543647+0000","last_clean":"2023-10-16T08:16:29.543647+0000","last_became_active":"2023-10-16T08:16:27.018428+0000","last_became_peered":"2023-10-16T08:16:27.018428+0000","last_unstale":"2023-10-16T08:16:29.543647+0000","last_undegraded":"2023-10-16T08:16:29.543647+0000","last_fullsized":"2023-10-16T08:16:29.543647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988140+0000","last_change":"2023-10-16T08:16:22.974536+0000","last_active":"2023-10-16T08:16:26.988140+0000","last_peered":"2023-10-16T08:16:26.988140+0000","last_clean":"2023-10-16T08:16:26.988140+0000","last_became_active":"2023-10-16T08:16:22.973582+0000","last_became_peered":"2023-10-16T08:16:22.973582+0000","last_unstale":"2023-10-16T08:16:26.988140+0000","last_undegraded":"2023-10-16T08:16:26.988140+0000","last_fullsized":"2023-10-16T08:16:26.988140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995774+0000","last_change":"2023-10-16T08:16:21.960308+0000","last_active":"2023-10-16T08:16:26.995774+0000","last_peered":"2023-10-16T08:16:26.995774+0000","last_clean":"2023-10-16T08:16:26.995774+0000","last_became_active":"2023-10-16T08:16:21.959328+0000","last_became_peered":"2023-10-16T08:16:21.959328+0000","last_unstale":"2023-10-16T08:16:26.995774+0000","last_undegraded":"2023-10-16T08:16:26.995774+0000","last_fullsized":"2023-10-16T08:16:26.995774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:15.899771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989677+0000","last_change":"2023-10-16T08:16:27.007716+0000","last_active":"2023-10-16T08:16:27.989677+0000","last_peered":"2023-10-16T08:16:27.989677+0000","last_clean":"2023-10-16T08:16:27.989677+0000","last_became_active":"2023-10-16T08:16:27.006885+0000","last_became_peered":"2023-10-16T08:16:27.006885+0000","last_unstale":"2023-10-16T08:16:27.989677+0000","last_undegraded":"2023-10-16T08:16:27.989677+0000","last_fullsized":"2023-10-16T08:16:27.989677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:18.625895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995685+0000","last_change":"2023-10-16T08:16:22.990790+0000","last_active":"2023-10-16T08:16:26.995685+0000","last_peered":"2023-10-16T08:16:26.995685+0000","last_clean":"2023-10-16T08:16:26.995685+0000","last_became_active":"2023-10-16T08:16:22.987169+0000","last_became_peered":"2023-10-16T08:16:22.987169+0000","last_unstale":"2023-10-16T08:16:26.995685+0000","last_undegraded":"2023-10-16T08:16:26.995685+0000","last_fullsized":"2023-10-16T08:16:26.995685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988440+0000","last_change":"2023-10-16T08:16:21.956681+0000","last_active":"2023-10-16T08:16:26.988440+0000","last_peered":"2023-10-16T08:16:26.988440+0000","last_clean":"2023-10-16T08:16:26.988440+0000","last_became_active":"2023-10-16T08:16:21.956318+0000","last_became_peered":"2023-10-16T08:16:21.956318+0000","last_unstale":"2023-10-16T08:16:26.988440+0000","last_undegraded":"2023-10-16T08:16:26.988440+0000","last_fullsized":"2023-10-16T08:16:26.988440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989667+0000","last_change":"2023-10-16T08:16:27.181126+0000","last_active":"2023-10-16T08:16:27.989667+0000","last_peered":"2023-10-16T08:16:27.989667+0000","last_clean":"2023-10-16T08:16:27.989667+0000","last_became_active":"2023-10-16T08:16:27.180722+0000","last_became_peered":"2023-10-16T08:16:27.180722+0000","last_unstale":"2023-10-16T08:16:27.989667+0000","last_undegraded":"2023-10-16T08:16:27.989667+0000","last_fullsized":"2023-10-16T08:16:27.989667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:39:03.047765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541176+0000","last_change":"2023-10-16T08:16:27.183947+0000","last_active":"2023-10-16T08:16:29.541176+0000","last_peered":"2023-10-16T08:16:29.541176+0000","last_clean":"2023-10-16T08:16:29.541176+0000","last_became_active":"2023-10-16T08:16:27.183487+0000","last_became_peered":"2023-10-16T08:16:27.183487+0000","last_unstale":"2023-10-16T08:16:29.541176+0000","last_undegraded":"2023-10-16T08:16:29.541176+0000","last_fullsized":"2023-10-16T08:16:29.541176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009520+0000","last_change":"2023-10-16T08:16:22.958791+0000","last_active":"2023-10-16T08:16:27.009520+0000","last_peered":"2023-10-16T08:16:27.009520+0000","last_clean":"2023-10-16T08:16:27.009520+0000","last_became_active":"2023-10-16T08:16:22.958513+0000","last_became_peered":"2023-10-16T08:16:22.958513+0000","last_unstale":"2023-10-16T08:16:27.009520+0000","last_undegraded":"2023-10-16T08:16:27.009520+0000","last_fullsized":"2023-10-16T08:16:27.009520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994423+0000","last_change":"2023-10-16T08:16:21.954384+0000","last_active":"2023-10-16T08:16:26.994423+0000","last_peered":"2023-10-16T08:16:26.994423+0000","last_clean":"2023-10-16T08:16:26.994423+0000","last_became_active":"2023-10-16T08:16:21.953995+0000","last_became_peered":"2023-10-16T08:16:21.953995+0000","last_unstale":"2023-10-16T08:16:26.994423+0000","last_undegraded":"2023-10-16T08:16:26.994423+0000","last_fullsized":"2023-10-16T08:16:26.994423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:34.686144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.992231+0000","last_change":"2023-10-16T08:16:27.008716+0000","last_active":"2023-10-16T08:16:27.992231+0000","last_peered":"2023-10-16T08:16:27.992231+0000","last_clean":"2023-10-16T08:16:27.992231+0000","last_became_active":"2023-10-16T08:16:27.007942+0000","last_became_peered":"2023-10-16T08:16:27.007942+0000","last_unstale":"2023-10-16T08:16:27.992231+0000","last_undegraded":"2023-10-16T08:16:27.992231+0000","last_fullsized":"2023-10-16T08:16:27.992231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986408+0000","last_change":"2023-10-16T08:16:22.969567+0000","last_active":"2023-10-16T08:16:26.986408+0000","last_peered":"2023-10-16T08:16:26.986408+0000","last_clean":"2023-10-16T08:16:26.986408+0000","last_became_active":"2023-10-16T08:16:22.969172+0000","last_became_peered":"2023-10-16T08:16:22.969172+0000","last_unstale":"2023-10-16T08:16:26.986408+0000","last_undegraded":"2023-10-16T08:16:26.986408+0000","last_fullsized":"2023-10-16T08:16:26.986408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986465+0000","last_change":"2023-10-16T08:16:21.956899+0000","last_active":"2023-10-16T08:16:26.986465+0000","last_peered":"2023-10-16T08:16:26.986465+0000","last_clean":"2023-10-16T08:16:26.986465+0000","last_became_active":"2023-10-16T08:16:21.956446+0000","last_became_peered":"2023-10-16T08:16:21.956446+0000","last_unstale":"2023-10-16T08:16:26.986465+0000","last_undegraded":"2023-10-16T08:16:26.986465+0000","last_fullsized":"2023-10-16T08:16:26.986465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987260+0000","last_change":"2023-10-16T08:16:21.955151+0000","last_active":"2023-10-16T08:16:26.987260+0000","last_peered":"2023-10-16T08:16:26.987260+0000","last_clean":"2023-10-16T08:16:26.987260+0000","last_became_active":"2023-10-16T08:16:21.953524+0000","last_became_peered":"2023-10-16T08:16:21.953524+0000","last_unstale":"2023-10-16T08:16:26.987260+0000","last_undegraded":"2023-10-16T08:16:26.987260+0000","last_fullsized":"2023-10-16T08:16:26.987260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988088+0000","last_change":"2023-10-16T08:16:22.965640+0000","last_active":"2023-10-16T08:16:26.988088+0000","last_peered":"2023-10-16T08:16:26.988088+0000","last_clean":"2023-10-16T08:16:26.988088+0000","last_became_active":"2023-10-16T08:16:22.963142+0000","last_became_peered":"2023-10-16T08:16:22.963142+0000","last_unstale":"2023-10-16T08:16:26.988088+0000","last_undegraded":"2023-10-16T08:16:26.988088+0000","last_fullsized":"2023-10-16T08:16:26.988088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.540964+0000","last_change":"2023-10-16T08:16:27.020258+0000","last_active":"2023-10-16T08:16:29.540964+0000","last_peered":"2023-10-16T08:16:29.540964+0000","last_clean":"2023-10-16T08:16:29.540964+0000","last_became_active":"2023-10-16T08:16:27.019846+0000","last_became_peered":"2023-10-16T08:16:27.019846+0000","last_unstale":"2023-10-16T08:16:29.540964+0000","last_undegraded":"2023-10-16T08:16:29.540964+0000","last_fullsized":"2023-10-16T08:16:29.540964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:47.301298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990459+0000","last_change":"2023-10-16T08:16:21.959137+0000","last_active":"2023-10-16T08:16:26.990459+0000","last_peered":"2023-10-16T08:16:26.990459+0000","last_clean":"2023-10-16T08:16:26.990459+0000","last_became_active":"2023-10-16T08:16:21.958785+0000","last_became_peered":"2023-10-16T08:16:21.958785+0000","last_unstale":"2023-10-16T08:16:26.990459+0000","last_undegraded":"2023-10-16T08:16:26.990459+0000","last_fullsized":"2023-10-16T08:16:26.990459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.999013+0000","last_change":"2023-10-16T08:16:22.961411+0000","last_active":"2023-10-16T08:16:26.999013+0000","last_peered":"2023-10-16T08:16:26.999013+0000","last_clean":"2023-10-16T08:16:26.999013+0000","last_became_active":"2023-10-16T08:16:22.961004+0000","last_became_peered":"2023-10-16T08:16:22.961004+0000","last_unstale":"2023-10-16T08:16:26.999013+0000","last_undegraded":"2023-10-16T08:16:26.999013+0000","last_fullsized":"2023-10-16T08:16:26.999013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:11.319467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990247+0000","last_change":"2023-10-16T08:16:27.014631+0000","last_active":"2023-10-16T08:16:27.990247+0000","last_peered":"2023-10-16T08:16:27.990247+0000","last_clean":"2023-10-16T08:16:27.990247+0000","last_became_active":"2023-10-16T08:16:27.014246+0000","last_became_peered":"2023-10-16T08:16:27.014246+0000","last_unstale":"2023-10-16T08:16:27.990247+0000","last_undegraded":"2023-10-16T08:16:27.990247+0000","last_fullsized":"2023-10-16T08:16:27.990247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:43.540909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986773+0000","last_change":"2023-10-16T08:16:22.971903+0000","last_active":"2023-10-16T08:16:26.986773+0000","last_peered":"2023-10-16T08:16:26.986773+0000","last_clean":"2023-10-16T08:16:26.986773+0000","last_became_active":"2023-10-16T08:16:22.971522+0000","last_became_peered":"2023-10-16T08:16:22.971522+0000","last_unstale":"2023-10-16T08:16:26.986773+0000","last_undegraded":"2023-10-16T08:16:26.986773+0000","last_fullsized":"2023-10-16T08:16:26.986773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996570+0000","last_change":"2023-10-16T08:16:22.970072+0000","last_active":"2023-10-16T08:16:26.996570+0000","last_peered":"2023-10-16T08:16:26.996570+0000","last_clean":"2023-10-16T08:16:26.996570+0000","last_became_active":"2023-10-16T08:16:22.969751+0000","last_became_peered":"2023-10-16T08:16:22.969751+0000","last_unstale":"2023-10-16T08:16:26.996570+0000","last_undegraded":"2023-10-16T08:16:26.996570+0000","last_fullsized":"2023-10-16T08:16:26.996570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994134+0000","last_change":"2023-10-16T08:16:22.995816+0000","last_active":"2023-10-16T08:16:26.994134+0000","last_peered":"2023-10-16T08:16:26.994134+0000","last_clean":"2023-10-16T08:16:26.994134+0000","last_became_active":"2023-10-16T08:16:22.990930+0000","last_became_peered":"2023-10-16T08:16:22.990930+0000","last_unstale":"2023-10-16T08:16:26.994134+0000","last_undegraded":"2023-10-16T08:16:26.994134+0000","last_fullsized":"2023-10-16T08:16:26.994134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987998+0000","last_change":"2023-10-16T08:16:22.972789+0000","last_active":"2023-10-16T08:16:26.987998+0000","last_peered":"2023-10-16T08:16:26.987998+0000","last_clean":"2023-10-16T08:16:26.987998+0000","last_became_active":"2023-10-16T08:16:22.972473+0000","last_became_peered":"2023-10-16T08:16:22.972473+0000","last_unstale":"2023-10-16T08:16:26.987998+0000","last_undegraded":"2023-10-16T08:16:26.987998+0000","last_fullsized":"2023-10-16T08:16:26.987998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985911+0000","last_change":"2023-10-16T08:16:22.971086+0000","last_active":"2023-10-16T08:16:26.985911+0000","last_peered":"2023-10-16T08:16:26.985911+0000","last_clean":"2023-10-16T08:16:26.985911+0000","last_became_active":"2023-10-16T08:16:22.962675+0000","last_became_peered":"2023-10-16T08:16:22.962675+0000","last_unstale":"2023-10-16T08:16:26.985911+0000","last_undegraded":"2023-10-16T08:16:26.985911+0000","last_fullsized":"2023-10-16T08:16:26.985911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013810+0000","last_change":"2023-10-16T08:16:22.962278+0000","last_active":"2023-10-16T08:16:27.013810+0000","last_peered":"2023-10-16T08:16:27.013810+0000","last_clean":"2023-10-16T08:16:27.013810+0000","last_became_active":"2023-10-16T08:16:22.962006+0000","last_became_peered":"2023-10-16T08:16:22.962006+0000","last_unstale":"2023-10-16T08:16:27.013810+0000","last_undegraded":"2023-10-16T08:16:27.013810+0000","last_fullsized":"2023-10-16T08:16:27.013810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975064+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.974692+0000","last_became_peered":"2023-10-16T08:16:22.974692+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987037+0000","last_change":"2023-10-16T08:16:22.961194+0000","last_active":"2023-10-16T08:16:26.987037+0000","last_peered":"2023-10-16T08:16:26.987037+0000","last_clean":"2023-10-16T08:16:26.987037+0000","last_became_active":"2023-10-16T08:16:22.960884+0000","last_became_peered":"2023-10-16T08:16:22.960884+0000","last_unstale":"2023-10-16T08:16:26.987037+0000","last_undegraded":"2023-10-16T08:16:26.987037+0000","last_fullsized":"2023-10-16T08:16:26.987037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994288+0000","last_change":"2023-10-16T08:16:22.983850+0000","last_active":"2023-10-16T08:16:26.994288+0000","last_peered":"2023-10-16T08:16:26.994288+0000","last_clean":"2023-10-16T08:16:26.994288+0000","last_became_active":"2023-10-16T08:16:22.978686+0000","last_became_peered":"2023-10-16T08:16:22.978686+0000","last_unstale":"2023-10-16T08:16:26.994288+0000","last_undegraded":"2023-10-16T08:16:26.994288+0000","last_fullsized":"2023-10-16T08:16:26.994288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986300+0000","last_change":"2023-10-16T08:16:22.957405+0000","last_active":"2023-10-16T08:16:26.986300+0000","last_peered":"2023-10-16T08:16:26.986300+0000","last_clean":"2023-10-16T08:16:26.986300+0000","last_became_active":"2023-10-16T08:16:22.957105+0000","last_became_peered":"2023-10-16T08:16:22.957105+0000","last_unstale":"2023-10-16T08:16:26.986300+0000","last_undegraded":"2023-10-16T08:16:26.986300+0000","last_fullsized":"2023-10-16T08:16:26.986300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994045+0000","last_change":"2023-10-16T08:16:22.985035+0000","last_active":"2023-10-16T08:16:26.994045+0000","last_peered":"2023-10-16T08:16:26.994045+0000","last_clean":"2023-10-16T08:16:26.994045+0000","last_became_active":"2023-10-16T08:16:22.984750+0000","last_became_peered":"2023-10-16T08:16:22.984750+0000","last_unstale":"2023-10-16T08:16:26.994045+0000","last_undegraded":"2023-10-16T08:16:26.994045+0000","last_fullsized":"2023-10-16T08:16:26.994045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987250+0000","last_change":"2023-10-16T08:16:22.976322+0000","last_active":"2023-10-16T08:16:26.987250+0000","last_peered":"2023-10-16T08:16:26.987250+0000","last_clean":"2023-10-16T08:16:26.987250+0000","last_became_active":"2023-10-16T08:16:22.976064+0000","last_became_peered":"2023-10-16T08:16:22.976064+0000","last_unstale":"2023-10-16T08:16:26.987250+0000","last_undegraded":"2023-10-16T08:16:26.987250+0000","last_fullsized":"2023-10-16T08:16:26.987250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987111+0000","last_change":"2023-10-16T08:16:22.965222+0000","last_active":"2023-10-16T08:16:26.987111+0000","last_peered":"2023-10-16T08:16:26.987111+0000","last_clean":"2023-10-16T08:16:26.987111+0000","last_became_active":"2023-10-16T08:16:22.964839+0000","last_became_peered":"2023-10-16T08:16:22.964839+0000","last_unstale":"2023-10-16T08:16:26.987111+0000","last_undegraded":"2023-10-16T08:16:26.987111+0000","last_fullsized":"2023-10-16T08:16:26.987111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.978846+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.975261+0000","last_became_peered":"2023-10-16T08:16:22.975261+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996150+0000","last_change":"2023-10-16T08:16:22.973907+0000","last_active":"2023-10-16T08:16:26.996150+0000","last_peered":"2023-10-16T08:16:26.996150+0000","last_clean":"2023-10-16T08:16:26.996150+0000","last_became_active":"2023-10-16T08:16:22.973609+0000","last_became_peered":"2023-10-16T08:16:22.973609+0000","last_unstale":"2023-10-16T08:16:26.996150+0000","last_undegraded":"2023-10-16T08:16:26.996150+0000","last_fullsized":"2023-10-16T08:16:26.996150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994986+0000","last_change":"2023-10-16T08:16:22.995822+0000","last_active":"2023-10-16T08:16:26.994986+0000","last_peered":"2023-10-16T08:16:26.994986+0000","last_clean":"2023-10-16T08:16:26.994986+0000","last_became_active":"2023-10-16T08:16:22.995473+0000","last_became_peered":"2023-10-16T08:16:22.995473+0000","last_unstale":"2023-10-16T08:16:26.994986+0000","last_undegraded":"2023-10-16T08:16:26.994986+0000","last_fullsized":"2023-10-16T08:16:26.994986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994188+0000","last_change":"2023-10-16T08:16:22.975976+0000","last_active":"2023-10-16T08:16:26.994188+0000","last_peered":"2023-10-16T08:16:26.994188+0000","last_clean":"2023-10-16T08:16:26.994188+0000","last_became_active":"2023-10-16T08:16:22.975600+0000","last_became_peered":"2023-10-16T08:16:22.975600+0000","last_unstale":"2023-10-16T08:16:26.994188+0000","last_undegraded":"2023-10-16T08:16:26.994188+0000","last_fullsized":"2023-10-16T08:16:26.994188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009586+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.009586+0000","last_peered":"2023-10-16T08:16:27.009586+0000","last_clean":"2023-10-16T08:16:27.009586+0000","last_became_active":"2023-10-16T08:16:22.954170+0000","last_became_peered":"2023-10-16T08:16:22.954170+0000","last_unstale":"2023-10-16T08:16:27.009586+0000","last_undegraded":"2023-10-16T08:16:27.009586+0000","last_fullsized":"2023-10-16T08:16:27.009586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984253+0000","last_change":"2023-10-16T08:16:22.959029+0000","last_active":"2023-10-16T08:16:26.984253+0000","last_peered":"2023-10-16T08:16:26.984253+0000","last_clean":"2023-10-16T08:16:26.984253+0000","last_became_active":"2023-10-16T08:16:22.958713+0000","last_became_peered":"2023-10-16T08:16:22.958713+0000","last_unstale":"2023-10-16T08:16:26.984253+0000","last_undegraded":"2023-10-16T08:16:26.984253+0000","last_fullsized":"2023-10-16T08:16:26.984253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:47.145986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990281+0000","last_change":"2023-10-16T08:16:22.973255+0000","last_active":"2023-10-16T08:16:26.990281+0000","last_peered":"2023-10-16T08:16:26.990281+0000","last_clean":"2023-10-16T08:16:26.990281+0000","last_became_active":"2023-10-16T08:16:22.972582+0000","last_became_peered":"2023-10-16T08:16:22.972582+0000","last_unstale":"2023-10-16T08:16:26.990281+0000","last_undegraded":"2023-10-16T08:16:26.990281+0000","last_fullsized":"2023-10-16T08:16:26.990281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004704+0000","last_change":"2023-10-16T08:16:22.969190+0000","last_active":"2023-10-16T08:16:27.004704+0000","last_peered":"2023-10-16T08:16:27.004704+0000","last_clean":"2023-10-16T08:16:27.004704+0000","last_became_active":"2023-10-16T08:16:22.968880+0000","last_became_peered":"2023-10-16T08:16:22.968880+0000","last_unstale":"2023-10-16T08:16:27.004704+0000","last_undegraded":"2023-10-16T08:16:27.004704+0000","last_fullsized":"2023-10-16T08:16:27.004704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987745+0000","last_change":"2023-10-16T08:16:22.975463+0000","last_active":"2023-10-16T08:16:26.987745+0000","last_peered":"2023-10-16T08:16:26.987745+0000","last_clean":"2023-10-16T08:16:26.987745+0000","last_became_active":"2023-10-16T08:16:22.975080+0000","last_became_peered":"2023-10-16T08:16:22.975080+0000","last_unstale":"2023-10-16T08:16:26.987745+0000","last_undegraded":"2023-10-16T08:16:26.987745+0000","last_fullsized":"2023-10-16T08:16:26.987745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975707+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.975429+0000","last_became_peered":"2023-10-16T08:16:22.975429+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179066+0000","last_change":"2023-10-16T08:16:22.977921+0000","last_active":"2023-10-16T08:16:27.179066+0000","last_peered":"2023-10-16T08:16:27.179066+0000","last_clean":"2023-10-16T08:16:27.179066+0000","last_became_active":"2023-10-16T08:16:22.974938+0000","last_became_peered":"2023-10-16T08:16:22.974938+0000","last_unstale":"2023-10-16T08:16:27.179066+0000","last_undegraded":"2023-10-16T08:16:27.179066+0000","last_fullsized":"2023-10-16T08:16:27.179066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:35.197545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986251+0000","last_change":"2023-10-16T08:16:22.972026+0000","last_active":"2023-10-16T08:16:26.986251+0000","last_peered":"2023-10-16T08:16:26.986251+0000","last_clean":"2023-10-16T08:16:26.986251+0000","last_became_active":"2023-10-16T08:16:22.971655+0000","last_became_peered":"2023-10-16T08:16:22.971655+0000","last_unstale":"2023-10-16T08:16:26.986251+0000","last_undegraded":"2023-10-16T08:16:26.986251+0000","last_fullsized":"2023-10-16T08:16:26.986251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178078+0000","last_change":"2023-10-16T08:16:22.973377+0000","last_active":"2023-10-16T08:16:27.178078+0000","last_peered":"2023-10-16T08:16:27.178078+0000","last_clean":"2023-10-16T08:16:27.178078+0000","last_became_active":"2023-10-16T08:16:22.972888+0000","last_became_peered":"2023-10-16T08:16:22.972888+0000","last_unstale":"2023-10-16T08:16:27.178078+0000","last_undegraded":"2023-10-16T08:16:27.178078+0000","last_fullsized":"2023-10-16T08:16:27.178078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994140+0000","last_change":"2023-10-16T08:16:22.976999+0000","last_active":"2023-10-16T08:16:26.994140+0000","last_peered":"2023-10-16T08:16:26.994140+0000","last_clean":"2023-10-16T08:16:26.994140+0000","last_became_active":"2023-10-16T08:16:22.974943+0000","last_became_peered":"2023-10-16T08:16:22.974943+0000","last_unstale":"2023-10-16T08:16:26.994140+0000","last_undegraded":"2023-10-16T08:16:26.994140+0000","last_fullsized":"2023-10-16T08:16:26.994140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178298+0000","last_change":"2023-10-16T08:16:22.979562+0000","last_active":"2023-10-16T08:16:27.178298+0000","last_peered":"2023-10-16T08:16:27.178298+0000","last_clean":"2023-10-16T08:16:27.178298+0000","last_became_active":"2023-10-16T08:16:22.975468+0000","last_became_peered":"2023-10-16T08:16:22.975468+0000","last_unstale":"2023-10-16T08:16:27.178298+0000","last_undegraded":"2023-10-16T08:16:27.178298+0000","last_fullsized":"2023-10-16T08:16:27.178298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994788+0000","last_change":"2023-10-16T08:16:22.991415+0000","last_active":"2023-10-16T08:16:26.994788+0000","last_peered":"2023-10-16T08:16:26.994788+0000","last_clean":"2023-10-16T08:16:26.994788+0000","last_became_active":"2023-10-16T08:16:22.990410+0000","last_became_peered":"2023-10-16T08:16:22.990410+0000","last_unstale":"2023-10-16T08:16:26.994788+0000","last_undegraded":"2023-10-16T08:16:26.994788+0000","last_fullsized":"2023-10-16T08:16:26.994788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987345+0000","last_change":"2023-10-16T08:16:22.957927+0000","last_active":"2023-10-16T08:16:26.987345+0000","last_peered":"2023-10-16T08:16:26.987345+0000","last_clean":"2023-10-16T08:16:26.987345+0000","last_became_active":"2023-10-16T08:16:22.956909+0000","last_became_peered":"2023-10-16T08:16:22.956909+0000","last_unstale":"2023-10-16T08:16:26.987345+0000","last_undegraded":"2023-10-16T08:16:26.987345+0000","last_fullsized":"2023-10-16T08:16:26.987345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014243+0000","last_change":"2023-10-16T08:16:22.963419+0000","last_active":"2023-10-16T08:16:27.014243+0000","last_peered":"2023-10-16T08:16:27.014243+0000","last_clean":"2023-10-16T08:16:27.014243+0000","last_became_active":"2023-10-16T08:16:22.962692+0000","last_became_peered":"2023-10-16T08:16:22.962692+0000","last_unstale":"2023-10-16T08:16:27.014243+0000","last_undegraded":"2023-10-16T08:16:27.014243+0000","last_fullsized":"2023-10-16T08:16:27.014243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178991+0000","last_change":"2023-10-16T08:16:22.976621+0000","last_active":"2023-10-16T08:16:27.178991+0000","last_peered":"2023-10-16T08:16:27.178991+0000","last_clean":"2023-10-16T08:16:27.178991+0000","last_became_active":"2023-10-16T08:16:22.975391+0000","last_became_peered":"2023-10-16T08:16:22.975391+0000","last_unstale":"2023-10-16T08:16:27.178991+0000","last_undegraded":"2023-10-16T08:16:27.178991+0000","last_fullsized":"2023-10-16T08:16:27.178991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.706}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:16:52.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:16:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 Processor -- start 2023-10-16T08:16:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 -- start start 2023-10-16T08:16:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb63c15cb90 0x7fb63c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d15d0 0x7fb63c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0ce770 con 0x7fb63c0d3010 2023-10-16T08:16:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0d0600 con 0x7fb63c15cb90 2023-10-16T08:16:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb641246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0c6450 con 0x7fb63c0d15d0 2023-10-16T08:16:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb63c15cb90 0x7fb63c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb63c15cb90 0x7fb63c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45830/0 (socket says 172.21.15.32:45830) 2023-10-16T08:16:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 -- 172.21.15.32:0/3361603939 learned_addr learned my addr 172.21.15.32:0/3361603939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63b59e640 1 --2- 172.21.15.32:0/3361603939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d15d0 0x7fb63c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 -- 172.21.15.32:0/3361603939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d15d0 msgr2=0x7fb63c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 --2- 172.21.15.32:0/3361603939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d15d0 0x7fb63c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 -- 172.21.15.32:0/3361603939 >> v1:172.21.15.32:6789/0 conn(0x7fb63c0d3010 legacy=0x7fb63c0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63a59c640 1 -- 172.21.15.32:0/3361603939 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355436738 0 0) 0x7fb63c0ce770 con 0x7fb63c0d3010 2023-10-16T08:16:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.134+0000 7fb63ad9d640 1 -- 172.21.15.32:0/3361603939 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0d08a0 con 0x7fb63c15cb90 2023-10-16T08:16:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb63ad9d640 1 --2- 172.21.15.32:0/3361603939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c15cb90 0x7fb63c0cfb10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb62c006fd0 tx=0x7fb62c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2de06fb51d4271 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb63a59c640 1 -- 172.21.15.32:0/3361603939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0ab660 con 0x7fb63c15cb90 2023-10-16T08:16:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb63a59c640 1 -- 172.21.15.32:0/3361603939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0ab7c0 con 0x7fb63c15cb90 2023-10-16T08:16:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb63a59c640 1 -- 172.21.15.32:0/3361603939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0abb50 con 0x7fb63c15cb90 2023-10-16T08:16:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 -- 172.21.15.32:0/3361603939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c15cb90 msgr2=0x7fb63c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 --2- 172.21.15.32:0/3361603939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c15cb90 0x7fb63c0cfb10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb62c006fd0 tx=0x7fb62c01c460 comp rx=0 tx=0).stop 2023-10-16T08:16:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 -- 172.21.15.32:0/3361603939 shutdown_connections 2023-10-16T08:16:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 --2- 172.21.15.32:0/3361603939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d15d0 0x7fb63c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 --2- 172.21.15.32:0/3361603939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c15cb90 0x7fb63c0cfb10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 -- 172.21.15.32:0/3361603939 >> 172.21.15.32:0/3361603939 conn(0x7fb63c09bc50 msgr2=0x7fb63c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 -- 172.21.15.32:0/3361603939 shutdown_connections 2023-10-16T08:16:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.138+0000 7fb641246640 1 -- 172.21.15.32:0/3361603939 wait complete. 2023-10-16T08:16:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 Processor -- start 2023-10-16T08:16:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 -- start start 2023-10-16T08:16:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 0x7fb63c0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d3010 0x7fb63c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb63c15cb90 0x7fb63c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15c3d0 con 0x7fb63c0d15d0 2023-10-16T08:16:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15b060 con 0x7fb63c15cb90 2023-10-16T08:16:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15b330 con 0x7fb63c0d3010 2023-10-16T08:16:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 0x7fb63c0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 0x7fb63c0c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45842/0 (socket says 172.21.15.32:45842) 2023-10-16T08:16:53.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 -- 172.21.15.32:0/2332399161 learned_addr learned my addr 172.21.15.32:0/2332399161 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:53.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63b59e640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb63c15cb90 0x7fb63c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb633fff640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d3010 0x7fb63c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 -- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d3010 msgr2=0x7fb63c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d3010 0x7fb63c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 -- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb63c15cb90 msgr2=0x7fb63c06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb63c15cb90 0x7fb63c06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 -- 172.21.15.32:0/2332399161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c009480 con 0x7fb63c0d15d0 2023-10-16T08:16:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb63ad9d640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 0x7fb63c0c9d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb62c01ab10 tx=0x7fb62c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0abd40 con 0x7fb63c0d15d0 2023-10-16T08:16:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0ab620 con 0x7fb63c0d15d0 2023-10-16T08:16:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c15b610 con 0x7fb63c0d15d0 2023-10-16T08:16:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0abea0 con 0x7fb63c0d15d0 2023-10-16T08:16:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.142+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c15bbd0 con 0x7fb63c0d15d0 2023-10-16T08:16:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.146+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c074010 con 0x7fb63c0d15d0 2023-10-16T08:16:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.146+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb62c0a9070 con 0x7fb63c0d15d0 2023-10-16T08:16:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.146+0000 7fb6337fe640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb61c0664d0 0x7fb61c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.146+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb62c11eea0 con 0x7fb63c0d15d0 2023-10-16T08:16:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.146+0000 7fb633fff640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb61c0664d0 0x7fb61c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.146+0000 7fb633fff640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb61c0664d0 0x7fb61c068990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0cc590 tx=0x7fb624009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.150+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb62c0ec0c0 con 0x7fb63c0d15d0 2023-10-16T08:16:53.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.306+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb63c15cf70 con 0x7fb61c0664d0 2023-10-16T08:16:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.314+0000 7fb6337fe640 1 -- 172.21.15.32:0/2332399161 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345302 (secure 0 0 0) 0x7fb624001d20 con 0x7fb61c0664d0 2023-10-16T08:16:53.324 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:16:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb61c0664d0 msgr2=0x7fb61c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb61c0664d0 0x7fb61c068990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0cc590 tx=0x7fb624009040 comp rx=0 tx=0).stop 2023-10-16T08:16:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 msgr2=0x7fb63c0c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 0x7fb63c0c9d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb62c01ab10 tx=0x7fb62c019200 comp rx=0 tx=0).stop 2023-10-16T08:16:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 shutdown_connections 2023-10-16T08:16:53.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb63c15cb90 0x7fb63c06b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb61c0664d0 0x7fb61c068990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb63c0d3010 0x7fb63c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 --2- 172.21.15.32:0/2332399161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb63c0d15d0 0x7fb63c0c9d70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 >> 172.21.15.32:0/2332399161 conn(0x7fb63c09bc50 msgr2=0x7fb63c159640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 shutdown_connections 2023-10-16T08:16:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.318+0000 7fb641246640 1 -- 172.21.15.32:0/2332399161 wait complete. 2023-10-16T08:16:53.344 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-16T08:16:52.021003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223628,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751092,"statfs":{"total":773094113280,"available":772865118208,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":225496635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003873"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988248+0000","last_change":"2023-10-16T08:16:21.956684+0000","last_active":"2023-10-16T08:16:26.988248+0000","last_peered":"2023-10-16T08:16:26.988248+0000","last_clean":"2023-10-16T08:16:26.988248+0000","last_became_active":"2023-10-16T08:16:21.949061+0000","last_became_peered":"2023-10-16T08:16:21.949061+0000","last_unstale":"2023-10-16T08:16:26.988248+0000","last_undegraded":"2023-10-16T08:16:26.988248+0000","last_fullsized":"2023-10-16T08:16:26.988248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985413+0000","last_change":"2023-10-16T08:16:22.972913+0000","last_active":"2023-10-16T08:16:26.985413+0000","last_peered":"2023-10-16T08:16:26.985413+0000","last_clean":"2023-10-16T08:16:26.985413+0000","last_became_active":"2023-10-16T08:16:22.972605+0000","last_became_peered":"2023-10-16T08:16:22.972605+0000","last_unstale":"2023-10-16T08:16:26.985413+0000","last_undegraded":"2023-10-16T08:16:26.985413+0000","last_fullsized":"2023-10-16T08:16:26.985413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989573+0000","last_change":"2023-10-16T08:16:27.182406+0000","last_active":"2023-10-16T08:16:27.989573+0000","last_peered":"2023-10-16T08:16:27.989573+0000","last_clean":"2023-10-16T08:16:27.989573+0000","last_became_active":"2023-10-16T08:16:27.182005+0000","last_became_peered":"2023-10-16T08:16:27.182005+0000","last_unstale":"2023-10-16T08:16:27.989573+0000","last_undegraded":"2023-10-16T08:16:27.989573+0000","last_fullsized":"2023-10-16T08:16:27.989573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:06.143173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985684+0000","last_change":"2023-10-16T08:16:21.945969+0000","last_active":"2023-10-16T08:16:26.985684+0000","last_peered":"2023-10-16T08:16:26.985684+0000","last_clean":"2023-10-16T08:16:26.985684+0000","last_became_active":"2023-10-16T08:16:21.945681+0000","last_became_peered":"2023-10-16T08:16:21.945681+0000","last_unstale":"2023-10-16T08:16:26.985684+0000","last_undegraded":"2023-10-16T08:16:26.985684+0000","last_fullsized":"2023-10-16T08:16:26.985684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988041+0000","last_change":"2023-10-16T08:16:22.974743+0000","last_active":"2023-10-16T08:16:26.988041+0000","last_peered":"2023-10-16T08:16:26.988041+0000","last_clean":"2023-10-16T08:16:26.988041+0000","last_became_active":"2023-10-16T08:16:22.974180+0000","last_became_peered":"2023-10-16T08:16:22.974180+0000","last_unstale":"2023-10-16T08:16:26.988041+0000","last_undegraded":"2023-10-16T08:16:26.988041+0000","last_fullsized":"2023-10-16T08:16:26.988041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542410+0000","last_change":"2023-10-16T08:16:27.011704+0000","last_active":"2023-10-16T08:16:29.542410+0000","last_peered":"2023-10-16T08:16:29.542410+0000","last_clean":"2023-10-16T08:16:29.542410+0000","last_became_active":"2023-10-16T08:16:27.011142+0000","last_became_peered":"2023-10-16T08:16:27.011142+0000","last_unstale":"2023-10-16T08:16:29.542410+0000","last_undegraded":"2023-10-16T08:16:29.542410+0000","last_fullsized":"2023-10-16T08:16:29.542410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:17.770482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178836+0000","last_change":"2023-10-16T08:16:21.964716+0000","last_active":"2023-10-16T08:16:27.178836+0000","last_peered":"2023-10-16T08:16:27.178836+0000","last_clean":"2023-10-16T08:16:27.178836+0000","last_became_active":"2023-10-16T08:16:21.964451+0000","last_became_peered":"2023-10-16T08:16:21.964451+0000","last_unstale":"2023-10-16T08:16:27.178836+0000","last_undegraded":"2023-10-16T08:16:27.178836+0000","last_fullsized":"2023-10-16T08:16:27.178836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994868+0000","last_change":"2023-10-16T08:16:22.971839+0000","last_active":"2023-10-16T08:16:26.994868+0000","last_peered":"2023-10-16T08:16:26.994868+0000","last_clean":"2023-10-16T08:16:26.994868+0000","last_became_active":"2023-10-16T08:16:22.971456+0000","last_became_peered":"2023-10-16T08:16:22.971456+0000","last_unstale":"2023-10-16T08:16:26.994868+0000","last_undegraded":"2023-10-16T08:16:26.994868+0000","last_fullsized":"2023-10-16T08:16:26.994868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543415+0000","last_change":"2023-10-16T08:16:27.024297+0000","last_active":"2023-10-16T08:16:29.543415+0000","last_peered":"2023-10-16T08:16:29.543415+0000","last_clean":"2023-10-16T08:16:29.543415+0000","last_became_active":"2023-10-16T08:16:27.023792+0000","last_became_peered":"2023-10-16T08:16:27.023792+0000","last_unstale":"2023-10-16T08:16:29.543415+0000","last_undegraded":"2023-10-16T08:16:29.543415+0000","last_fullsized":"2023-10-16T08:16:29.543415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:20.519605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178617+0000","last_change":"2023-10-16T08:16:21.964025+0000","last_active":"2023-10-16T08:16:27.178617+0000","last_peered":"2023-10-16T08:16:27.178617+0000","last_clean":"2023-10-16T08:16:27.178617+0000","last_became_active":"2023-10-16T08:16:21.963582+0000","last_became_peered":"2023-10-16T08:16:21.963582+0000","last_unstale":"2023-10-16T08:16:27.178617+0000","last_undegraded":"2023-10-16T08:16:27.178617+0000","last_fullsized":"2023-10-16T08:16:27.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994775+0000","last_change":"2023-10-16T08:16:22.980735+0000","last_active":"2023-10-16T08:16:26.994775+0000","last_peered":"2023-10-16T08:16:26.994775+0000","last_clean":"2023-10-16T08:16:26.994775+0000","last_became_active":"2023-10-16T08:16:22.980409+0000","last_became_peered":"2023-10-16T08:16:22.980409+0000","last_unstale":"2023-10-16T08:16:26.994775+0000","last_undegraded":"2023-10-16T08:16:26.994775+0000","last_fullsized":"2023-10-16T08:16:26.994775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542893+0000","last_change":"2023-10-16T08:16:27.024269+0000","last_active":"2023-10-16T08:16:29.542893+0000","last_peered":"2023-10-16T08:16:29.542893+0000","last_clean":"2023-10-16T08:16:29.542893+0000","last_became_active":"2023-10-16T08:16:27.023822+0000","last_became_peered":"2023-10-16T08:16:27.023822+0000","last_unstale":"2023-10-16T08:16:29.542893+0000","last_undegraded":"2023-10-16T08:16:29.542893+0000","last_fullsized":"2023-10-16T08:16:29.542893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:49.956062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987114+0000","last_change":"2023-10-16T08:16:22.969401+0000","last_active":"2023-10-16T08:16:26.987114+0000","last_peered":"2023-10-16T08:16:26.987114+0000","last_clean":"2023-10-16T08:16:26.987114+0000","last_became_active":"2023-10-16T08:16:22.969099+0000","last_became_peered":"2023-10-16T08:16:22.969099+0000","last_unstale":"2023-10-16T08:16:26.987114+0000","last_undegraded":"2023-10-16T08:16:26.987114+0000","last_fullsized":"2023-10-16T08:16:26.987114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178720+0000","last_change":"2023-10-16T08:16:21.961388+0000","last_active":"2023-10-16T08:16:27.178720+0000","last_peered":"2023-10-16T08:16:27.178720+0000","last_clean":"2023-10-16T08:16:27.178720+0000","last_became_active":"2023-10-16T08:16:21.961050+0000","last_became_peered":"2023-10-16T08:16:21.961050+0000","last_unstale":"2023-10-16T08:16:27.178720+0000","last_undegraded":"2023-10-16T08:16:27.178720+0000","last_fullsized":"2023-10-16T08:16:27.178720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:13.387501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543302+0000","last_change":"2023-10-16T08:16:27.019072+0000","last_active":"2023-10-16T08:16:29.543302+0000","last_peered":"2023-10-16T08:16:29.543302+0000","last_clean":"2023-10-16T08:16:29.543302+0000","last_became_active":"2023-10-16T08:16:27.018440+0000","last_became_peered":"2023-10-16T08:16:27.018440+0000","last_unstale":"2023-10-16T08:16:29.543302+0000","last_undegraded":"2023-10-16T08:16:29.543302+0000","last_fullsized":"2023-10-16T08:16:29.543302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:39.310251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990373+0000","last_change":"2023-10-16T08:16:22.960667+0000","last_active":"2023-10-16T08:16:26.990373+0000","last_peered":"2023-10-16T08:16:26.990373+0000","last_clean":"2023-10-16T08:16:26.990373+0000","last_became_active":"2023-10-16T08:16:22.960345+0000","last_became_peered":"2023-10-16T08:16:22.960345+0000","last_unstale":"2023-10-16T08:16:26.990373+0000","last_undegraded":"2023-10-16T08:16:26.990373+0000","last_fullsized":"2023-10-16T08:16:26.990373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013495+0000","last_change":"2023-10-16T08:16:21.950408+0000","last_active":"2023-10-16T08:16:27.013495+0000","last_peered":"2023-10-16T08:16:27.013495+0000","last_clean":"2023-10-16T08:16:27.013495+0000","last_became_active":"2023-10-16T08:16:21.947942+0000","last_became_peered":"2023-10-16T08:16:21.947942+0000","last_unstale":"2023-10-16T08:16:27.013495+0000","last_undegraded":"2023-10-16T08:16:27.013495+0000","last_fullsized":"2023-10-16T08:16:27.013495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542836+0000","last_change":"2023-10-16T08:16:27.021686+0000","last_active":"2023-10-16T08:16:29.542836+0000","last_peered":"2023-10-16T08:16:29.542836+0000","last_clean":"2023-10-16T08:16:29.542836+0000","last_became_active":"2023-10-16T08:16:27.021230+0000","last_became_peered":"2023-10-16T08:16:27.021230+0000","last_unstale":"2023-10-16T08:16:29.542836+0000","last_undegraded":"2023-10-16T08:16:29.542836+0000","last_fullsized":"2023-10-16T08:16:29.542836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:12.583792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986505+0000","last_change":"2023-10-16T08:16:22.955965+0000","last_active":"2023-10-16T08:16:26.986505+0000","last_peered":"2023-10-16T08:16:26.986505+0000","last_clean":"2023-10-16T08:16:26.986505+0000","last_became_active":"2023-10-16T08:16:22.955652+0000","last_became_peered":"2023-10-16T08:16:22.955652+0000","last_unstale":"2023-10-16T08:16:26.986505+0000","last_undegraded":"2023-10-16T08:16:26.986505+0000","last_fullsized":"2023-10-16T08:16:26.986505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009595+0000","last_change":"2023-10-16T08:16:21.950219+0000","last_active":"2023-10-16T08:16:27.009595+0000","last_peered":"2023-10-16T08:16:27.009595+0000","last_clean":"2023-10-16T08:16:27.009595+0000","last_became_active":"2023-10-16T08:16:21.949749+0000","last_became_peered":"2023-10-16T08:16:21.949749+0000","last_unstale":"2023-10-16T08:16:27.009595+0000","last_undegraded":"2023-10-16T08:16:27.009595+0000","last_fullsized":"2023-10-16T08:16:27.009595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988141+0000","last_change":"2023-10-16T08:16:27.182341+0000","last_active":"2023-10-16T08:16:27.988141+0000","last_peered":"2023-10-16T08:16:27.988141+0000","last_clean":"2023-10-16T08:16:27.988141+0000","last_became_active":"2023-10-16T08:16:27.181854+0000","last_became_peered":"2023-10-16T08:16:27.181854+0000","last_unstale":"2023-10-16T08:16:27.988141+0000","last_undegraded":"2023-10-16T08:16:27.988141+0000","last_fullsized":"2023-10-16T08:16:27.988141+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003825+0000","last_change":"2023-10-16T08:16:22.962471+0000","last_active":"2023-10-16T08:16:27.003825+0000","last_peered":"2023-10-16T08:16:27.003825+0000","last_clean":"2023-10-16T08:16:27.003825+0000","last_became_active":"2023-10-16T08:16:22.961915+0000","last_became_peered":"2023-10-16T08:16:22.961915+0000","last_unstale":"2023-10-16T08:16:27.003825+0000","last_undegraded":"2023-10-16T08:16:27.003825+0000","last_fullsized":"2023-10-16T08:16:27.003825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:13.023207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177873+0000","last_change":"2023-10-16T08:16:21.964429+0000","last_active":"2023-10-16T08:16:27.177873+0000","last_peered":"2023-10-16T08:16:27.177873+0000","last_clean":"2023-10-16T08:16:27.177873+0000","last_became_active":"2023-10-16T08:16:21.964154+0000","last_became_peered":"2023-10-16T08:16:21.964154+0000","last_unstale":"2023-10-16T08:16:27.177873+0000","last_undegraded":"2023-10-16T08:16:27.177873+0000","last_fullsized":"2023-10-16T08:16:27.177873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543629+0000","last_change":"2023-10-16T08:16:27.027359+0000","last_active":"2023-10-16T08:16:29.543629+0000","last_peered":"2023-10-16T08:16:29.543629+0000","last_clean":"2023-10-16T08:16:29.543629+0000","last_became_active":"2023-10-16T08:16:27.026923+0000","last_became_peered":"2023-10-16T08:16:27.026923+0000","last_unstale":"2023-10-16T08:16:29.543629+0000","last_undegraded":"2023-10-16T08:16:29.543629+0000","last_fullsized":"2023-10-16T08:16:29.543629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.989245+0000","last_change":"2023-10-16T08:16:22.967177+0000","last_active":"2023-10-16T08:16:26.989245+0000","last_peered":"2023-10-16T08:16:26.989245+0000","last_clean":"2023-10-16T08:16:26.989245+0000","last_became_active":"2023-10-16T08:16:22.966812+0000","last_became_peered":"2023-10-16T08:16:22.966812+0000","last_unstale":"2023-10-16T08:16:26.989245+0000","last_undegraded":"2023-10-16T08:16:26.989245+0000","last_fullsized":"2023-10-16T08:16:26.989245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988555+0000","last_change":"2023-10-16T08:16:21.958338+0000","last_active":"2023-10-16T08:16:26.988555+0000","last_peered":"2023-10-16T08:16:26.988555+0000","last_clean":"2023-10-16T08:16:26.988555+0000","last_became_active":"2023-10-16T08:16:21.958052+0000","last_became_peered":"2023-10-16T08:16:21.958052+0000","last_unstale":"2023-10-16T08:16:26.988555+0000","last_undegraded":"2023-10-16T08:16:26.988555+0000","last_fullsized":"2023-10-16T08:16:26.988555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543530+0000","last_change":"2023-10-16T08:16:27.184138+0000","last_active":"2023-10-16T08:16:29.543530+0000","last_peered":"2023-10-16T08:16:29.543530+0000","last_clean":"2023-10-16T08:16:29.543530+0000","last_became_active":"2023-10-16T08:16:27.183769+0000","last_became_peered":"2023-10-16T08:16:27.183769+0000","last_unstale":"2023-10-16T08:16:29.543530+0000","last_undegraded":"2023-10-16T08:16:29.543530+0000","last_fullsized":"2023-10-16T08:16:29.543530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987117+0000","last_change":"2023-10-16T08:16:22.973708+0000","last_active":"2023-10-16T08:16:26.987117+0000","last_peered":"2023-10-16T08:16:26.987117+0000","last_clean":"2023-10-16T08:16:26.987117+0000","last_became_active":"2023-10-16T08:16:22.973402+0000","last_became_peered":"2023-10-16T08:16:22.973402+0000","last_unstale":"2023-10-16T08:16:26.987117+0000","last_undegraded":"2023-10-16T08:16:26.987117+0000","last_fullsized":"2023-10-16T08:16:26.987117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014342+0000","last_change":"2023-10-16T08:16:21.955020+0000","last_active":"2023-10-16T08:16:27.014342+0000","last_peered":"2023-10-16T08:16:27.014342+0000","last_clean":"2023-10-16T08:16:27.014342+0000","last_became_active":"2023-10-16T08:16:21.954681+0000","last_became_peered":"2023-10-16T08:16:21.954681+0000","last_unstale":"2023-10-16T08:16:27.014342+0000","last_undegraded":"2023-10-16T08:16:27.014342+0000","last_fullsized":"2023-10-16T08:16:27.014342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989486+0000","last_change":"2023-10-16T08:16:27.179578+0000","last_active":"2023-10-16T08:16:27.989486+0000","last_peered":"2023-10-16T08:16:27.989486+0000","last_clean":"2023-10-16T08:16:27.989486+0000","last_became_active":"2023-10-16T08:16:27.178683+0000","last_became_peered":"2023-10-16T08:16:27.178683+0000","last_unstale":"2023-10-16T08:16:27.989486+0000","last_undegraded":"2023-10-16T08:16:27.989486+0000","last_fullsized":"2023-10-16T08:16:27.989486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014216+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.014216+0000","last_peered":"2023-10-16T08:16:27.014216+0000","last_clean":"2023-10-16T08:16:27.014216+0000","last_became_active":"2023-10-16T08:16:22.952824+0000","last_became_peered":"2023-10-16T08:16:22.952824+0000","last_unstale":"2023-10-16T08:16:27.014216+0000","last_undegraded":"2023-10-16T08:16:27.014216+0000","last_fullsized":"2023-10-16T08:16:27.014216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178889+0000","last_change":"2023-10-16T08:16:21.964665+0000","last_active":"2023-10-16T08:16:27.178889+0000","last_peered":"2023-10-16T08:16:27.178889+0000","last_clean":"2023-10-16T08:16:27.178889+0000","last_became_active":"2023-10-16T08:16:21.964105+0000","last_became_peered":"2023-10-16T08:16:21.964105+0000","last_unstale":"2023-10-16T08:16:27.178889+0000","last_undegraded":"2023-10-16T08:16:27.178889+0000","last_fullsized":"2023-10-16T08:16:27.178889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:12.709836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541103+0000","last_change":"2023-10-16T08:16:27.180634+0000","last_active":"2023-10-16T08:16:29.541103+0000","last_peered":"2023-10-16T08:16:29.541103+0000","last_clean":"2023-10-16T08:16:29.541103+0000","last_became_active":"2023-10-16T08:16:27.180186+0000","last_became_peered":"2023-10-16T08:16:27.180186+0000","last_unstale":"2023-10-16T08:16:29.541103+0000","last_undegraded":"2023-10-16T08:16:29.541103+0000","last_fullsized":"2023-10-16T08:16:29.541103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994880+0000","last_change":"2023-10-16T08:16:22.994702+0000","last_active":"2023-10-16T08:16:26.994880+0000","last_peered":"2023-10-16T08:16:26.994880+0000","last_clean":"2023-10-16T08:16:26.994880+0000","last_became_active":"2023-10-16T08:16:22.990441+0000","last_became_peered":"2023-10-16T08:16:22.990441+0000","last_unstale":"2023-10-16T08:16:26.994880+0000","last_undegraded":"2023-10-16T08:16:26.994880+0000","last_fullsized":"2023-10-16T08:16:26.994880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014301+0000","last_change":"2023-10-16T08:16:21.942771+0000","last_active":"2023-10-16T08:16:27.014301+0000","last_peered":"2023-10-16T08:16:27.014301+0000","last_clean":"2023-10-16T08:16:27.014301+0000","last_became_active":"2023-10-16T08:16:21.942441+0000","last_became_peered":"2023-10-16T08:16:21.942441+0000","last_unstale":"2023-10-16T08:16:27.014301+0000","last_undegraded":"2023-10-16T08:16:27.014301+0000","last_fullsized":"2023-10-16T08:16:27.014301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.986815+0000","last_change":"2023-10-16T08:16:27.013697+0000","last_active":"2023-10-16T08:16:27.986815+0000","last_peered":"2023-10-16T08:16:27.986815+0000","last_clean":"2023-10-16T08:16:27.986815+0000","last_became_active":"2023-10-16T08:16:27.012937+0000","last_became_peered":"2023-10-16T08:16:27.012937+0000","last_unstale":"2023-10-16T08:16:27.986815+0000","last_undegraded":"2023-10-16T08:16:27.986815+0000","last_fullsized":"2023-10-16T08:16:27.986815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:29.729911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.003417+0000","last_change":"2023-10-16T08:16:22.964069+0000","last_active":"2023-10-16T08:16:27.003417+0000","last_peered":"2023-10-16T08:16:27.003417+0000","last_clean":"2023-10-16T08:16:27.003417+0000","last_became_active":"2023-10-16T08:16:22.963802+0000","last_became_peered":"2023-10-16T08:16:22.963802+0000","last_unstale":"2023-10-16T08:16:27.003417+0000","last_undegraded":"2023-10-16T08:16:27.003417+0000","last_fullsized":"2023-10-16T08:16:27.003417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994257+0000","last_change":"2023-10-16T08:16:21.962674+0000","last_active":"2023-10-16T08:16:26.994257+0000","last_peered":"2023-10-16T08:16:26.994257+0000","last_clean":"2023-10-16T08:16:26.994257+0000","last_became_active":"2023-10-16T08:16:21.962316+0000","last_became_peered":"2023-10-16T08:16:21.962316+0000","last_unstale":"2023-10-16T08:16:26.994257+0000","last_undegraded":"2023-10-16T08:16:26.994257+0000","last_fullsized":"2023-10-16T08:16:26.994257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:32.032856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987533+0000","last_change":"2023-10-16T08:16:27.020747+0000","last_active":"2023-10-16T08:16:27.987533+0000","last_peered":"2023-10-16T08:16:27.987533+0000","last_clean":"2023-10-16T08:16:27.987533+0000","last_became_active":"2023-10-16T08:16:27.019634+0000","last_became_peered":"2023-10-16T08:16:27.019634+0000","last_unstale":"2023-10-16T08:16:27.987533+0000","last_undegraded":"2023-10-16T08:16:27.987533+0000","last_fullsized":"2023-10-16T08:16:27.987533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009729+0000","last_change":"2023-10-16T08:16:22.953894+0000","last_active":"2023-10-16T08:16:27.009729+0000","last_peered":"2023-10-16T08:16:27.009729+0000","last_clean":"2023-10-16T08:16:27.009729+0000","last_became_active":"2023-10-16T08:16:22.953606+0000","last_became_peered":"2023-10-16T08:16:22.953606+0000","last_unstale":"2023-10-16T08:16:27.009729+0000","last_undegraded":"2023-10-16T08:16:27.009729+0000","last_fullsized":"2023-10-16T08:16:27.009729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996654+0000","last_change":"2023-10-16T08:16:21.946996+0000","last_active":"2023-10-16T08:16:26.996654+0000","last_peered":"2023-10-16T08:16:26.996654+0000","last_clean":"2023-10-16T08:16:26.996654+0000","last_became_active":"2023-10-16T08:16:21.946675+0000","last_became_peered":"2023-10-16T08:16:21.946675+0000","last_unstale":"2023-10-16T08:16:26.996654+0000","last_undegraded":"2023-10-16T08:16:26.996654+0000","last_fullsized":"2023-10-16T08:16:26.996654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:43:55.303584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.988192+0000","last_change":"2023-10-16T08:16:27.020112+0000","last_active":"2023-10-16T08:16:27.988192+0000","last_peered":"2023-10-16T08:16:27.988192+0000","last_clean":"2023-10-16T08:16:27.988192+0000","last_became_active":"2023-10-16T08:16:27.019606+0000","last_became_peered":"2023-10-16T08:16:27.019606+0000","last_unstale":"2023-10-16T08:16:27.988192+0000","last_undegraded":"2023-10-16T08:16:27.988192+0000","last_fullsized":"2023-10-16T08:16:27.988192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984195+0000","last_change":"2023-10-16T08:16:22.962655+0000","last_active":"2023-10-16T08:16:26.984195+0000","last_peered":"2023-10-16T08:16:26.984195+0000","last_clean":"2023-10-16T08:16:26.984195+0000","last_became_active":"2023-10-16T08:16:22.962344+0000","last_became_peered":"2023-10-16T08:16:22.962344+0000","last_unstale":"2023-10-16T08:16:26.984195+0000","last_undegraded":"2023-10-16T08:16:26.984195+0000","last_fullsized":"2023-10-16T08:16:26.984195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:43.247033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177937+0000","last_change":"2023-10-16T08:16:21.959357+0000","last_active":"2023-10-16T08:16:27.177937+0000","last_peered":"2023-10-16T08:16:27.177937+0000","last_clean":"2023-10-16T08:16:27.177937+0000","last_became_active":"2023-10-16T08:16:21.959015+0000","last_became_peered":"2023-10-16T08:16:21.959015+0000","last_unstale":"2023-10-16T08:16:27.177937+0000","last_undegraded":"2023-10-16T08:16:27.177937+0000","last_fullsized":"2023-10-16T08:16:27.177937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:02.223178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541384+0000","last_change":"2023-10-16T08:16:27.179195+0000","last_active":"2023-10-16T08:16:29.541384+0000","last_peered":"2023-10-16T08:16:29.541384+0000","last_clean":"2023-10-16T08:16:29.541384+0000","last_became_active":"2023-10-16T08:16:27.178732+0000","last_became_peered":"2023-10-16T08:16:27.178732+0000","last_unstale":"2023-10-16T08:16:29.541384+0000","last_undegraded":"2023-10-16T08:16:29.541384+0000","last_fullsized":"2023-10-16T08:16:29.541384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:15.036715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986752+0000","last_change":"2023-10-16T08:16:22.973605+0000","last_active":"2023-10-16T08:16:26.986752+0000","last_peered":"2023-10-16T08:16:26.986752+0000","last_clean":"2023-10-16T08:16:26.986752+0000","last_became_active":"2023-10-16T08:16:22.973281+0000","last_became_peered":"2023-10-16T08:16:22.973281+0000","last_unstale":"2023-10-16T08:16:26.986752+0000","last_undegraded":"2023-10-16T08:16:26.986752+0000","last_fullsized":"2023-10-16T08:16:26.986752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994407+0000","last_change":"2023-10-16T08:16:21.950850+0000","last_active":"2023-10-16T08:16:26.994407+0000","last_peered":"2023-10-16T08:16:26.994407+0000","last_clean":"2023-10-16T08:16:26.994407+0000","last_became_active":"2023-10-16T08:16:21.950526+0000","last_became_peered":"2023-10-16T08:16:21.950526+0000","last_unstale":"2023-10-16T08:16:26.994407+0000","last_undegraded":"2023-10-16T08:16:26.994407+0000","last_fullsized":"2023-10-16T08:16:26.994407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:00.893346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.177966+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:16:27.177966+0000","last_peered":"2023-10-16T08:16:27.177966+0000","last_clean":"2023-10-16T08:16:27.177966+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:16:27.177966+0000","last_undegraded":"2023-10-16T08:16:27.177966+0000","last_fullsized":"2023-10-16T08:16:27.177966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991969+0000","last_change":"2023-10-16T08:16:27.184250+0000","last_active":"2023-10-16T08:16:27.991969+0000","last_peered":"2023-10-16T08:16:27.991969+0000","last_clean":"2023-10-16T08:16:27.991969+0000","last_became_active":"2023-10-16T08:16:27.183826+0000","last_became_peered":"2023-10-16T08:16:27.183826+0000","last_unstale":"2023-10-16T08:16:27.991969+0000","last_undegraded":"2023-10-16T08:16:27.991969+0000","last_fullsized":"2023-10-16T08:16:27.991969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:55.447764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178713+0000","last_change":"2023-10-16T08:16:22.975303+0000","last_active":"2023-10-16T08:16:27.178713+0000","last_peered":"2023-10-16T08:16:27.178713+0000","last_clean":"2023-10-16T08:16:27.178713+0000","last_became_active":"2023-10-16T08:16:22.974890+0000","last_became_peered":"2023-10-16T08:16:22.974890+0000","last_unstale":"2023-10-16T08:16:27.178713+0000","last_undegraded":"2023-10-16T08:16:27.178713+0000","last_fullsized":"2023-10-16T08:16:27.178713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:51.393499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.997283+0000","last_change":"2023-10-16T08:16:21.959337+0000","last_active":"2023-10-16T08:16:26.997283+0000","last_peered":"2023-10-16T08:16:26.997283+0000","last_clean":"2023-10-16T08:16:26.997283+0000","last_became_active":"2023-10-16T08:16:21.959059+0000","last_became_peered":"2023-10-16T08:16:21.959059+0000","last_unstale":"2023-10-16T08:16:26.997283+0000","last_undegraded":"2023-10-16T08:16:26.997283+0000","last_fullsized":"2023-10-16T08:16:26.997283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.991525+0000","last_change":"2023-10-16T08:16:27.181051+0000","last_active":"2023-10-16T08:16:27.991525+0000","last_peered":"2023-10-16T08:16:27.991525+0000","last_clean":"2023-10-16T08:16:27.991525+0000","last_became_active":"2023-10-16T08:16:27.180503+0000","last_became_peered":"2023-10-16T08:16:27.180503+0000","last_unstale":"2023-10-16T08:16:27.991525+0000","last_undegraded":"2023-10-16T08:16:27.991525+0000","last_fullsized":"2023-10-16T08:16:27.991525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:53.880783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994965+0000","last_change":"2023-10-16T08:16:22.990070+0000","last_active":"2023-10-16T08:16:26.994965+0000","last_peered":"2023-10-16T08:16:26.994965+0000","last_clean":"2023-10-16T08:16:26.994965+0000","last_became_active":"2023-10-16T08:16:22.989787+0000","last_became_peered":"2023-10-16T08:16:22.989787+0000","last_unstale":"2023-10-16T08:16:26.994965+0000","last_undegraded":"2023-10-16T08:16:26.994965+0000","last_fullsized":"2023-10-16T08:16:26.994965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013982+0000","last_change":"2023-10-16T08:16:21.961287+0000","last_active":"2023-10-16T08:16:27.013982+0000","last_peered":"2023-10-16T08:16:27.013982+0000","last_clean":"2023-10-16T08:16:27.013982+0000","last_became_active":"2023-10-16T08:16:21.956293+0000","last_became_peered":"2023-10-16T08:16:21.956293+0000","last_unstale":"2023-10-16T08:16:27.013982+0000","last_undegraded":"2023-10-16T08:16:27.013982+0000","last_fullsized":"2023-10-16T08:16:27.013982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:56:49.044714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541257+0000","last_change":"2023-10-16T08:16:27.013097+0000","last_active":"2023-10-16T08:16:29.541257+0000","last_peered":"2023-10-16T08:16:29.541257+0000","last_clean":"2023-10-16T08:16:29.541257+0000","last_became_active":"2023-10-16T08:16:27.012587+0000","last_became_peered":"2023-10-16T08:16:27.012587+0000","last_unstale":"2023-10-16T08:16:29.541257+0000","last_undegraded":"2023-10-16T08:16:29.541257+0000","last_fullsized":"2023-10-16T08:16:29.541257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:39.251810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179067+0000","last_change":"2023-10-16T08:16:22.979584+0000","last_active":"2023-10-16T08:16:27.179067+0000","last_peered":"2023-10-16T08:16:27.179067+0000","last_clean":"2023-10-16T08:16:27.179067+0000","last_became_active":"2023-10-16T08:16:22.978239+0000","last_became_peered":"2023-10-16T08:16:22.978239+0000","last_unstale":"2023-10-16T08:16:27.179067+0000","last_undegraded":"2023-10-16T08:16:27.179067+0000","last_fullsized":"2023-10-16T08:16:27.179067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:27:57.200006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996247+0000","last_change":"2023-10-16T08:16:21.947037+0000","last_active":"2023-10-16T08:16:26.996247+0000","last_peered":"2023-10-16T08:16:26.996247+0000","last_clean":"2023-10-16T08:16:26.996247+0000","last_became_active":"2023-10-16T08:16:21.946671+0000","last_became_peered":"2023-10-16T08:16:21.946671+0000","last_unstale":"2023-10-16T08:16:26.996247+0000","last_undegraded":"2023-10-16T08:16:26.996247+0000","last_fullsized":"2023-10-16T08:16:26.996247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.542197+0000","last_change":"2023-10-16T08:16:27.189477+0000","last_active":"2023-10-16T08:16:29.542197+0000","last_peered":"2023-10-16T08:16:29.542197+0000","last_clean":"2023-10-16T08:16:29.542197+0000","last_became_active":"2023-10-16T08:16:27.185942+0000","last_became_peered":"2023-10-16T08:16:27.185942+0000","last_unstale":"2023-10-16T08:16:29.542197+0000","last_undegraded":"2023-10-16T08:16:29.542197+0000","last_fullsized":"2023-10-16T08:16:29.542197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:13.645047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.973600+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.973215+0000","last_became_peered":"2023-10-16T08:16:22.973215+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:25.675646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.005005+0000","last_change":"2023-10-16T08:16:21.949416+0000","last_active":"2023-10-16T08:16:27.005005+0000","last_peered":"2023-10-16T08:16:27.005005+0000","last_clean":"2023-10-16T08:16:27.005005+0000","last_became_active":"2023-10-16T08:16:21.948836+0000","last_became_peered":"2023-10-16T08:16:21.948836+0000","last_unstale":"2023-10-16T08:16:27.005005+0000","last_undegraded":"2023-10-16T08:16:27.005005+0000","last_fullsized":"2023-10-16T08:16:27.005005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:56.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543245+0000","last_change":"2023-10-16T08:16:27.012879+0000","last_active":"2023-10-16T08:16:29.543245+0000","last_peered":"2023-10-16T08:16:29.543245+0000","last_clean":"2023-10-16T08:16:29.543245+0000","last_became_active":"2023-10-16T08:16:27.012466+0000","last_became_peered":"2023-10-16T08:16:27.012466+0000","last_unstale":"2023-10-16T08:16:29.543245+0000","last_undegraded":"2023-10-16T08:16:29.543245+0000","last_fullsized":"2023-10-16T08:16:29.543245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:16.486175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178093+0000","last_change":"2023-10-16T08:16:22.973193+0000","last_active":"2023-10-16T08:16:27.178093+0000","last_peered":"2023-10-16T08:16:27.178093+0000","last_clean":"2023-10-16T08:16:27.178093+0000","last_became_active":"2023-10-16T08:16:22.972472+0000","last_became_peered":"2023-10-16T08:16:22.972472+0000","last_unstale":"2023-10-16T08:16:27.178093+0000","last_undegraded":"2023-10-16T08:16:27.178093+0000","last_fullsized":"2023-10-16T08:16:27.178093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:35.255985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996389+0000","last_change":"2023-10-16T08:16:21.952668+0000","last_active":"2023-10-16T08:16:26.996389+0000","last_peered":"2023-10-16T08:16:26.996389+0000","last_clean":"2023-10-16T08:16:26.996389+0000","last_became_active":"2023-10-16T08:16:21.952006+0000","last_became_peered":"2023-10-16T08:16:21.952006+0000","last_unstale":"2023-10-16T08:16:26.996389+0000","last_undegraded":"2023-10-16T08:16:26.996389+0000","last_fullsized":"2023-10-16T08:16:26.996389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:12.016826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541359+0000","last_change":"2023-10-16T08:16:27.015675+0000","last_active":"2023-10-16T08:16:29.541359+0000","last_peered":"2023-10-16T08:16:29.541359+0000","last_clean":"2023-10-16T08:16:29.541359+0000","last_became_active":"2023-10-16T08:16:27.015279+0000","last_became_peered":"2023-10-16T08:16:27.015279+0000","last_unstale":"2023-10-16T08:16:29.541359+0000","last_undegraded":"2023-10-16T08:16:29.541359+0000","last_fullsized":"2023-10-16T08:16:29.541359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:01.254271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986776+0000","last_change":"2023-10-16T08:16:22.963194+0000","last_active":"2023-10-16T08:16:26.986776+0000","last_peered":"2023-10-16T08:16:26.986776+0000","last_clean":"2023-10-16T08:16:26.986776+0000","last_became_active":"2023-10-16T08:16:22.962589+0000","last_became_peered":"2023-10-16T08:16:22.962589+0000","last_unstale":"2023-10-16T08:16:26.986776+0000","last_undegraded":"2023-10-16T08:16:26.986776+0000","last_fullsized":"2023-10-16T08:16:26.986776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986872+0000","last_change":"2023-10-16T08:16:21.957192+0000","last_active":"2023-10-16T08:16:26.986872+0000","last_peered":"2023-10-16T08:16:26.986872+0000","last_clean":"2023-10-16T08:16:26.986872+0000","last_became_active":"2023-10-16T08:16:21.955564+0000","last_became_peered":"2023-10-16T08:16:21.955564+0000","last_unstale":"2023-10-16T08:16:26.986872+0000","last_undegraded":"2023-10-16T08:16:26.986872+0000","last_fullsized":"2023-10-16T08:16:26.986872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:58.308103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990800+0000","last_change":"2023-10-16T08:16:27.185040+0000","last_active":"2023-10-16T08:16:27.990800+0000","last_peered":"2023-10-16T08:16:27.990800+0000","last_clean":"2023-10-16T08:16:27.990800+0000","last_became_active":"2023-10-16T08:16:27.184448+0000","last_became_peered":"2023-10-16T08:16:27.184448+0000","last_unstale":"2023-10-16T08:16:27.990800+0000","last_undegraded":"2023-10-16T08:16:27.990800+0000","last_fullsized":"2023-10-16T08:16:27.990800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:59.852030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178532+0000","last_change":"2023-10-16T08:16:22.980031+0000","last_active":"2023-10-16T08:16:27.178532+0000","last_peered":"2023-10-16T08:16:27.178532+0000","last_clean":"2023-10-16T08:16:27.178532+0000","last_became_active":"2023-10-16T08:16:22.976708+0000","last_became_peered":"2023-10-16T08:16:22.976708+0000","last_unstale":"2023-10-16T08:16:27.178532+0000","last_undegraded":"2023-10-16T08:16:27.178532+0000","last_fullsized":"2023-10-16T08:16:27.178532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994242+0000","last_change":"2023-10-16T08:16:21.956986+0000","last_active":"2023-10-16T08:16:26.994242+0000","last_peered":"2023-10-16T08:16:26.994242+0000","last_clean":"2023-10-16T08:16:26.994242+0000","last_became_active":"2023-10-16T08:16:21.956307+0000","last_became_peered":"2023-10-16T08:16:21.956307+0000","last_unstale":"2023-10-16T08:16:26.994242+0000","last_undegraded":"2023-10-16T08:16:26.994242+0000","last_fullsized":"2023-10-16T08:16:26.994242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:26.142575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990801+0000","last_change":"2023-10-16T08:16:27.179150+0000","last_active":"2023-10-16T08:16:27.990801+0000","last_peered":"2023-10-16T08:16:27.990801+0000","last_clean":"2023-10-16T08:16:27.990801+0000","last_became_active":"2023-10-16T08:16:27.178649+0000","last_became_peered":"2023-10-16T08:16:27.178649+0000","last_unstale":"2023-10-16T08:16:27.990801+0000","last_undegraded":"2023-10-16T08:16:27.990801+0000","last_fullsized":"2023-10-16T08:16:27.990801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:45.567047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984308+0000","last_change":"2023-10-16T08:16:22.967505+0000","last_active":"2023-10-16T08:16:26.984308+0000","last_peered":"2023-10-16T08:16:26.984308+0000","last_clean":"2023-10-16T08:16:26.984308+0000","last_became_active":"2023-10-16T08:16:22.967214+0000","last_became_peered":"2023-10-16T08:16:22.967214+0000","last_unstale":"2023-10-16T08:16:26.984308+0000","last_undegraded":"2023-10-16T08:16:26.984308+0000","last_fullsized":"2023-10-16T08:16:26.984308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988187+0000","last_change":"2023-10-16T08:16:21.946248+0000","last_active":"2023-10-16T08:16:26.988187+0000","last_peered":"2023-10-16T08:16:26.988187+0000","last_clean":"2023-10-16T08:16:26.988187+0000","last_became_active":"2023-10-16T08:16:21.945958+0000","last_became_peered":"2023-10-16T08:16:21.945958+0000","last_unstale":"2023-10-16T08:16:26.988187+0000","last_undegraded":"2023-10-16T08:16:26.988187+0000","last_fullsized":"2023-10-16T08:16:26.988187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:10.313085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543632+0000","last_change":"2023-10-16T08:16:27.184504+0000","last_active":"2023-10-16T08:16:29.543632+0000","last_peered":"2023-10-16T08:16:29.543632+0000","last_clean":"2023-10-16T08:16:29.543632+0000","last_became_active":"2023-10-16T08:16:27.184029+0000","last_became_peered":"2023-10-16T08:16:27.184029+0000","last_unstale":"2023-10-16T08:16:29.543632+0000","last_undegraded":"2023-10-16T08:16:29.543632+0000","last_fullsized":"2023-10-16T08:16:29.543632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:30:37.313084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178733+0000","last_change":"2023-10-16T08:16:22.986066+0000","last_active":"2023-10-16T08:16:27.178733+0000","last_peered":"2023-10-16T08:16:27.178733+0000","last_clean":"2023-10-16T08:16:27.178733+0000","last_became_active":"2023-10-16T08:16:22.985791+0000","last_became_peered":"2023-10-16T08:16:22.985791+0000","last_unstale":"2023-10-16T08:16:27.178733+0000","last_undegraded":"2023-10-16T08:16:27.178733+0000","last_fullsized":"2023-10-16T08:16:27.178733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:38.000149+0000","last_change":"2023-10-16T08:16:21.955781+0000","last_active":"2023-10-16T08:16:38.000149+0000","last_peered":"2023-10-16T08:16:38.000149+0000","last_clean":"2023-10-16T08:16:38.000149+0000","last_became_active":"2023-10-16T08:16:21.955327+0000","last_became_peered":"2023-10-16T08:16:21.955327+0000","last_unstale":"2023-10-16T08:16:38.000149+0000","last_undegraded":"2023-10-16T08:16:38.000149+0000","last_fullsized":"2023-10-16T08:16:38.000149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.987157+0000","last_change":"2023-10-16T08:16:27.007137+0000","last_active":"2023-10-16T08:16:27.987157+0000","last_peered":"2023-10-16T08:16:27.987157+0000","last_clean":"2023-10-16T08:16:27.987157+0000","last_became_active":"2023-10-16T08:16:27.006722+0000","last_became_peered":"2023-10-16T08:16:27.006722+0000","last_unstale":"2023-10-16T08:16:27.987157+0000","last_undegraded":"2023-10-16T08:16:27.987157+0000","last_fullsized":"2023-10-16T08:16:27.987157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004799+0000","last_change":"2023-10-16T08:16:22.976159+0000","last_active":"2023-10-16T08:16:27.004799+0000","last_peered":"2023-10-16T08:16:27.004799+0000","last_clean":"2023-10-16T08:16:27.004799+0000","last_became_active":"2023-10-16T08:16:22.975879+0000","last_became_peered":"2023-10-16T08:16:22.975879+0000","last_unstale":"2023-10-16T08:16:27.004799+0000","last_undegraded":"2023-10-16T08:16:27.004799+0000","last_fullsized":"2023-10-16T08:16:27.004799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:04.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013901+0000","last_change":"2023-10-16T08:16:21.961500+0000","last_active":"2023-10-16T08:16:27.013901+0000","last_peered":"2023-10-16T08:16:27.013901+0000","last_clean":"2023-10-16T08:16:27.013901+0000","last_became_active":"2023-10-16T08:16:21.959870+0000","last_became_peered":"2023-10-16T08:16:21.959870+0000","last_unstale":"2023-10-16T08:16:27.013901+0000","last_undegraded":"2023-10-16T08:16:27.013901+0000","last_fullsized":"2023-10-16T08:16:27.013901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:57:03.429649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.543647+0000","last_change":"2023-10-16T08:16:27.018940+0000","last_active":"2023-10-16T08:16:29.543647+0000","last_peered":"2023-10-16T08:16:29.543647+0000","last_clean":"2023-10-16T08:16:29.543647+0000","last_became_active":"2023-10-16T08:16:27.018428+0000","last_became_peered":"2023-10-16T08:16:27.018428+0000","last_unstale":"2023-10-16T08:16:29.543647+0000","last_undegraded":"2023-10-16T08:16:29.543647+0000","last_fullsized":"2023-10-16T08:16:29.543647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988140+0000","last_change":"2023-10-16T08:16:22.974536+0000","last_active":"2023-10-16T08:16:26.988140+0000","last_peered":"2023-10-16T08:16:26.988140+0000","last_clean":"2023-10-16T08:16:26.988140+0000","last_became_active":"2023-10-16T08:16:22.973582+0000","last_became_peered":"2023-10-16T08:16:22.973582+0000","last_unstale":"2023-10-16T08:16:26.988140+0000","last_undegraded":"2023-10-16T08:16:26.988140+0000","last_fullsized":"2023-10-16T08:16:26.988140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995774+0000","last_change":"2023-10-16T08:16:21.960308+0000","last_active":"2023-10-16T08:16:26.995774+0000","last_peered":"2023-10-16T08:16:26.995774+0000","last_clean":"2023-10-16T08:16:26.995774+0000","last_became_active":"2023-10-16T08:16:21.959328+0000","last_became_peered":"2023-10-16T08:16:21.959328+0000","last_unstale":"2023-10-16T08:16:26.995774+0000","last_undegraded":"2023-10-16T08:16:26.995774+0000","last_fullsized":"2023-10-16T08:16:26.995774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:15.899771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989677+0000","last_change":"2023-10-16T08:16:27.007716+0000","last_active":"2023-10-16T08:16:27.989677+0000","last_peered":"2023-10-16T08:16:27.989677+0000","last_clean":"2023-10-16T08:16:27.989677+0000","last_became_active":"2023-10-16T08:16:27.006885+0000","last_became_peered":"2023-10-16T08:16:27.006885+0000","last_unstale":"2023-10-16T08:16:27.989677+0000","last_undegraded":"2023-10-16T08:16:27.989677+0000","last_fullsized":"2023-10-16T08:16:27.989677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:18.625895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.995685+0000","last_change":"2023-10-16T08:16:22.990790+0000","last_active":"2023-10-16T08:16:26.995685+0000","last_peered":"2023-10-16T08:16:26.995685+0000","last_clean":"2023-10-16T08:16:26.995685+0000","last_became_active":"2023-10-16T08:16:22.987169+0000","last_became_peered":"2023-10-16T08:16:22.987169+0000","last_unstale":"2023-10-16T08:16:26.995685+0000","last_undegraded":"2023-10-16T08:16:26.995685+0000","last_fullsized":"2023-10-16T08:16:26.995685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988440+0000","last_change":"2023-10-16T08:16:21.956681+0000","last_active":"2023-10-16T08:16:26.988440+0000","last_peered":"2023-10-16T08:16:26.988440+0000","last_clean":"2023-10-16T08:16:26.988440+0000","last_became_active":"2023-10-16T08:16:21.956318+0000","last_became_peered":"2023-10-16T08:16:21.956318+0000","last_unstale":"2023-10-16T08:16:26.988440+0000","last_undegraded":"2023-10-16T08:16:26.988440+0000","last_fullsized":"2023-10-16T08:16:26.988440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.989667+0000","last_change":"2023-10-16T08:16:27.181126+0000","last_active":"2023-10-16T08:16:27.989667+0000","last_peered":"2023-10-16T08:16:27.989667+0000","last_clean":"2023-10-16T08:16:27.989667+0000","last_became_active":"2023-10-16T08:16:27.180722+0000","last_became_peered":"2023-10-16T08:16:27.180722+0000","last_unstale":"2023-10-16T08:16:27.989667+0000","last_undegraded":"2023-10-16T08:16:27.989667+0000","last_fullsized":"2023-10-16T08:16:27.989667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:39:03.047765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.541176+0000","last_change":"2023-10-16T08:16:27.183947+0000","last_active":"2023-10-16T08:16:29.541176+0000","last_peered":"2023-10-16T08:16:29.541176+0000","last_clean":"2023-10-16T08:16:29.541176+0000","last_became_active":"2023-10-16T08:16:27.183487+0000","last_became_peered":"2023-10-16T08:16:27.183487+0000","last_unstale":"2023-10-16T08:16:29.541176+0000","last_undegraded":"2023-10-16T08:16:29.541176+0000","last_fullsized":"2023-10-16T08:16:29.541176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:11.158555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009520+0000","last_change":"2023-10-16T08:16:22.958791+0000","last_active":"2023-10-16T08:16:27.009520+0000","last_peered":"2023-10-16T08:16:27.009520+0000","last_clean":"2023-10-16T08:16:27.009520+0000","last_became_active":"2023-10-16T08:16:22.958513+0000","last_became_peered":"2023-10-16T08:16:22.958513+0000","last_unstale":"2023-10-16T08:16:27.009520+0000","last_undegraded":"2023-10-16T08:16:27.009520+0000","last_fullsized":"2023-10-16T08:16:27.009520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994423+0000","last_change":"2023-10-16T08:16:21.954384+0000","last_active":"2023-10-16T08:16:26.994423+0000","last_peered":"2023-10-16T08:16:26.994423+0000","last_clean":"2023-10-16T08:16:26.994423+0000","last_became_active":"2023-10-16T08:16:21.953995+0000","last_became_peered":"2023-10-16T08:16:21.953995+0000","last_unstale":"2023-10-16T08:16:26.994423+0000","last_undegraded":"2023-10-16T08:16:26.994423+0000","last_fullsized":"2023-10-16T08:16:26.994423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:34.686144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.992231+0000","last_change":"2023-10-16T08:16:27.008716+0000","last_active":"2023-10-16T08:16:27.992231+0000","last_peered":"2023-10-16T08:16:27.992231+0000","last_clean":"2023-10-16T08:16:27.992231+0000","last_became_active":"2023-10-16T08:16:27.007942+0000","last_became_peered":"2023-10-16T08:16:27.007942+0000","last_unstale":"2023-10-16T08:16:27.992231+0000","last_undegraded":"2023-10-16T08:16:27.992231+0000","last_fullsized":"2023-10-16T08:16:27.992231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:12.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986408+0000","last_change":"2023-10-16T08:16:22.969567+0000","last_active":"2023-10-16T08:16:26.986408+0000","last_peered":"2023-10-16T08:16:26.986408+0000","last_clean":"2023-10-16T08:16:26.986408+0000","last_became_active":"2023-10-16T08:16:22.969172+0000","last_became_peered":"2023-10-16T08:16:22.969172+0000","last_unstale":"2023-10-16T08:16:26.986408+0000","last_undegraded":"2023-10-16T08:16:26.986408+0000","last_fullsized":"2023-10-16T08:16:26.986408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:01.899727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986465+0000","last_change":"2023-10-16T08:16:21.956899+0000","last_active":"2023-10-16T08:16:26.986465+0000","last_peered":"2023-10-16T08:16:26.986465+0000","last_clean":"2023-10-16T08:16:26.986465+0000","last_became_active":"2023-10-16T08:16:21.956446+0000","last_became_peered":"2023-10-16T08:16:21.956446+0000","last_unstale":"2023-10-16T08:16:26.986465+0000","last_undegraded":"2023-10-16T08:16:26.986465+0000","last_fullsized":"2023-10-16T08:16:26.986465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:02.136482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987260+0000","last_change":"2023-10-16T08:16:21.955151+0000","last_active":"2023-10-16T08:16:26.987260+0000","last_peered":"2023-10-16T08:16:26.987260+0000","last_clean":"2023-10-16T08:16:26.987260+0000","last_became_active":"2023-10-16T08:16:21.953524+0000","last_became_peered":"2023-10-16T08:16:21.953524+0000","last_unstale":"2023-10-16T08:16:26.987260+0000","last_undegraded":"2023-10-16T08:16:26.987260+0000","last_fullsized":"2023-10-16T08:16:26.987260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:46.139605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.988088+0000","last_change":"2023-10-16T08:16:22.965640+0000","last_active":"2023-10-16T08:16:26.988088+0000","last_peered":"2023-10-16T08:16:26.988088+0000","last_clean":"2023-10-16T08:16:26.988088+0000","last_became_active":"2023-10-16T08:16:22.963142+0000","last_became_peered":"2023-10-16T08:16:22.963142+0000","last_unstale":"2023-10-16T08:16:26.988088+0000","last_undegraded":"2023-10-16T08:16:26.988088+0000","last_fullsized":"2023-10-16T08:16:26.988088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:29.540964+0000","last_change":"2023-10-16T08:16:27.020258+0000","last_active":"2023-10-16T08:16:29.540964+0000","last_peered":"2023-10-16T08:16:29.540964+0000","last_clean":"2023-10-16T08:16:29.540964+0000","last_became_active":"2023-10-16T08:16:27.019846+0000","last_became_peered":"2023-10-16T08:16:27.019846+0000","last_unstale":"2023-10-16T08:16:29.540964+0000","last_undegraded":"2023-10-16T08:16:29.540964+0000","last_fullsized":"2023-10-16T08:16:29.540964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:47.301298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990459+0000","last_change":"2023-10-16T08:16:21.959137+0000","last_active":"2023-10-16T08:16:26.990459+0000","last_peered":"2023-10-16T08:16:26.990459+0000","last_clean":"2023-10-16T08:16:26.990459+0000","last_became_active":"2023-10-16T08:16:21.958785+0000","last_became_peered":"2023-10-16T08:16:21.958785+0000","last_unstale":"2023-10-16T08:16:26.990459+0000","last_undegraded":"2023-10-16T08:16:26.990459+0000","last_fullsized":"2023-10-16T08:16:26.990459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:20.910367+0000","last_clean_scrub_stamp":"2023-10-16T08:16:20.910367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:06:42.240652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.999013+0000","last_change":"2023-10-16T08:16:22.961411+0000","last_active":"2023-10-16T08:16:26.999013+0000","last_peered":"2023-10-16T08:16:26.999013+0000","last_clean":"2023-10-16T08:16:26.999013+0000","last_became_active":"2023-10-16T08:16:22.961004+0000","last_became_peered":"2023-10-16T08:16:22.961004+0000","last_unstale":"2023-10-16T08:16:26.999013+0000","last_undegraded":"2023-10-16T08:16:26.999013+0000","last_fullsized":"2023-10-16T08:16:26.999013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:41:11.319467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.990247+0000","last_change":"2023-10-16T08:16:27.014631+0000","last_active":"2023-10-16T08:16:27.990247+0000","last_peered":"2023-10-16T08:16:27.990247+0000","last_clean":"2023-10-16T08:16:27.990247+0000","last_became_active":"2023-10-16T08:16:27.014246+0000","last_became_peered":"2023-10-16T08:16:27.014246+0000","last_unstale":"2023-10-16T08:16:27.990247+0000","last_undegraded":"2023-10-16T08:16:27.990247+0000","last_fullsized":"2023-10-16T08:16:27.990247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:25.971014+0000","last_clean_scrub_stamp":"2023-10-16T08:16:25.971014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:05:43.540909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986773+0000","last_change":"2023-10-16T08:16:22.971903+0000","last_active":"2023-10-16T08:16:26.986773+0000","last_peered":"2023-10-16T08:16:26.986773+0000","last_clean":"2023-10-16T08:16:26.986773+0000","last_became_active":"2023-10-16T08:16:22.971522+0000","last_became_peered":"2023-10-16T08:16:22.971522+0000","last_unstale":"2023-10-16T08:16:26.986773+0000","last_undegraded":"2023-10-16T08:16:26.986773+0000","last_fullsized":"2023-10-16T08:16:26.986773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996570+0000","last_change":"2023-10-16T08:16:22.970072+0000","last_active":"2023-10-16T08:16:26.996570+0000","last_peered":"2023-10-16T08:16:26.996570+0000","last_clean":"2023-10-16T08:16:26.996570+0000","last_became_active":"2023-10-16T08:16:22.969751+0000","last_became_peered":"2023-10-16T08:16:22.969751+0000","last_unstale":"2023-10-16T08:16:26.996570+0000","last_undegraded":"2023-10-16T08:16:26.996570+0000","last_fullsized":"2023-10-16T08:16:26.996570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:03.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994134+0000","last_change":"2023-10-16T08:16:22.995816+0000","last_active":"2023-10-16T08:16:26.994134+0000","last_peered":"2023-10-16T08:16:26.994134+0000","last_clean":"2023-10-16T08:16:26.994134+0000","last_became_active":"2023-10-16T08:16:22.990930+0000","last_became_peered":"2023-10-16T08:16:22.990930+0000","last_unstale":"2023-10-16T08:16:26.994134+0000","last_undegraded":"2023-10-16T08:16:26.994134+0000","last_fullsized":"2023-10-16T08:16:26.994134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987998+0000","last_change":"2023-10-16T08:16:22.972789+0000","last_active":"2023-10-16T08:16:26.987998+0000","last_peered":"2023-10-16T08:16:26.987998+0000","last_clean":"2023-10-16T08:16:26.987998+0000","last_became_active":"2023-10-16T08:16:22.972473+0000","last_became_peered":"2023-10-16T08:16:22.972473+0000","last_unstale":"2023-10-16T08:16:26.987998+0000","last_undegraded":"2023-10-16T08:16:26.987998+0000","last_fullsized":"2023-10-16T08:16:26.987998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.985911+0000","last_change":"2023-10-16T08:16:22.971086+0000","last_active":"2023-10-16T08:16:26.985911+0000","last_peered":"2023-10-16T08:16:26.985911+0000","last_clean":"2023-10-16T08:16:26.985911+0000","last_became_active":"2023-10-16T08:16:22.962675+0000","last_became_peered":"2023-10-16T08:16:22.962675+0000","last_unstale":"2023-10-16T08:16:26.985911+0000","last_undegraded":"2023-10-16T08:16:26.985911+0000","last_fullsized":"2023-10-16T08:16:26.985911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:16.465339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.013810+0000","last_change":"2023-10-16T08:16:22.962278+0000","last_active":"2023-10-16T08:16:27.013810+0000","last_peered":"2023-10-16T08:16:27.013810+0000","last_clean":"2023-10-16T08:16:27.013810+0000","last_became_active":"2023-10-16T08:16:22.962006+0000","last_became_peered":"2023-10-16T08:16:22.962006+0000","last_unstale":"2023-10-16T08:16:27.013810+0000","last_undegraded":"2023-10-16T08:16:27.013810+0000","last_fullsized":"2023-10-16T08:16:27.013810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975064+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.974692+0000","last_became_peered":"2023-10-16T08:16:22.974692+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987037+0000","last_change":"2023-10-16T08:16:22.961194+0000","last_active":"2023-10-16T08:16:26.987037+0000","last_peered":"2023-10-16T08:16:26.987037+0000","last_clean":"2023-10-16T08:16:26.987037+0000","last_became_active":"2023-10-16T08:16:22.960884+0000","last_became_peered":"2023-10-16T08:16:22.960884+0000","last_unstale":"2023-10-16T08:16:26.987037+0000","last_undegraded":"2023-10-16T08:16:26.987037+0000","last_fullsized":"2023-10-16T08:16:26.987037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994288+0000","last_change":"2023-10-16T08:16:22.983850+0000","last_active":"2023-10-16T08:16:26.994288+0000","last_peered":"2023-10-16T08:16:26.994288+0000","last_clean":"2023-10-16T08:16:26.994288+0000","last_became_active":"2023-10-16T08:16:22.978686+0000","last_became_peered":"2023-10-16T08:16:22.978686+0000","last_unstale":"2023-10-16T08:16:26.994288+0000","last_undegraded":"2023-10-16T08:16:26.994288+0000","last_fullsized":"2023-10-16T08:16:26.994288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986300+0000","last_change":"2023-10-16T08:16:22.957405+0000","last_active":"2023-10-16T08:16:26.986300+0000","last_peered":"2023-10-16T08:16:26.986300+0000","last_clean":"2023-10-16T08:16:26.986300+0000","last_became_active":"2023-10-16T08:16:22.957105+0000","last_became_peered":"2023-10-16T08:16:22.957105+0000","last_unstale":"2023-10-16T08:16:26.986300+0000","last_undegraded":"2023-10-16T08:16:26.986300+0000","last_fullsized":"2023-10-16T08:16:26.986300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994045+0000","last_change":"2023-10-16T08:16:22.985035+0000","last_active":"2023-10-16T08:16:26.994045+0000","last_peered":"2023-10-16T08:16:26.994045+0000","last_clean":"2023-10-16T08:16:26.994045+0000","last_became_active":"2023-10-16T08:16:22.984750+0000","last_became_peered":"2023-10-16T08:16:22.984750+0000","last_unstale":"2023-10-16T08:16:26.994045+0000","last_undegraded":"2023-10-16T08:16:26.994045+0000","last_fullsized":"2023-10-16T08:16:26.994045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987250+0000","last_change":"2023-10-16T08:16:22.976322+0000","last_active":"2023-10-16T08:16:26.987250+0000","last_peered":"2023-10-16T08:16:26.987250+0000","last_clean":"2023-10-16T08:16:26.987250+0000","last_became_active":"2023-10-16T08:16:22.976064+0000","last_became_peered":"2023-10-16T08:16:22.976064+0000","last_unstale":"2023-10-16T08:16:26.987250+0000","last_undegraded":"2023-10-16T08:16:26.987250+0000","last_fullsized":"2023-10-16T08:16:26.987250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987111+0000","last_change":"2023-10-16T08:16:22.965222+0000","last_active":"2023-10-16T08:16:26.987111+0000","last_peered":"2023-10-16T08:16:26.987111+0000","last_clean":"2023-10-16T08:16:26.987111+0000","last_became_active":"2023-10-16T08:16:22.964839+0000","last_became_peered":"2023-10-16T08:16:22.964839+0000","last_unstale":"2023-10-16T08:16:26.987111+0000","last_undegraded":"2023-10-16T08:16:26.987111+0000","last_fullsized":"2023-10-16T08:16:26.987111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178841+0000","last_change":"2023-10-16T08:16:22.978846+0000","last_active":"2023-10-16T08:16:27.178841+0000","last_peered":"2023-10-16T08:16:27.178841+0000","last_clean":"2023-10-16T08:16:27.178841+0000","last_became_active":"2023-10-16T08:16:22.975261+0000","last_became_peered":"2023-10-16T08:16:22.975261+0000","last_unstale":"2023-10-16T08:16:27.178841+0000","last_undegraded":"2023-10-16T08:16:27.178841+0000","last_fullsized":"2023-10-16T08:16:27.178841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:34:08.199519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.996150+0000","last_change":"2023-10-16T08:16:22.973907+0000","last_active":"2023-10-16T08:16:26.996150+0000","last_peered":"2023-10-16T08:16:26.996150+0000","last_clean":"2023-10-16T08:16:26.996150+0000","last_became_active":"2023-10-16T08:16:22.973609+0000","last_became_peered":"2023-10-16T08:16:22.973609+0000","last_unstale":"2023-10-16T08:16:26.996150+0000","last_undegraded":"2023-10-16T08:16:26.996150+0000","last_fullsized":"2023-10-16T08:16:26.996150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994986+0000","last_change":"2023-10-16T08:16:22.995822+0000","last_active":"2023-10-16T08:16:26.994986+0000","last_peered":"2023-10-16T08:16:26.994986+0000","last_clean":"2023-10-16T08:16:26.994986+0000","last_became_active":"2023-10-16T08:16:22.995473+0000","last_became_peered":"2023-10-16T08:16:22.995473+0000","last_unstale":"2023-10-16T08:16:26.994986+0000","last_undegraded":"2023-10-16T08:16:26.994986+0000","last_fullsized":"2023-10-16T08:16:26.994986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994188+0000","last_change":"2023-10-16T08:16:22.975976+0000","last_active":"2023-10-16T08:16:26.994188+0000","last_peered":"2023-10-16T08:16:26.994188+0000","last_clean":"2023-10-16T08:16:26.994188+0000","last_became_active":"2023-10-16T08:16:22.975600+0000","last_became_peered":"2023-10-16T08:16:22.975600+0000","last_unstale":"2023-10-16T08:16:26.994188+0000","last_undegraded":"2023-10-16T08:16:26.994188+0000","last_fullsized":"2023-10-16T08:16:26.994188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:14.393883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.009586+0000","last_change":"2023-10-16T08:16:22.954544+0000","last_active":"2023-10-16T08:16:27.009586+0000","last_peered":"2023-10-16T08:16:27.009586+0000","last_clean":"2023-10-16T08:16:27.009586+0000","last_became_active":"2023-10-16T08:16:22.954170+0000","last_became_peered":"2023-10-16T08:16:22.954170+0000","last_unstale":"2023-10-16T08:16:27.009586+0000","last_undegraded":"2023-10-16T08:16:27.009586+0000","last_fullsized":"2023-10-16T08:16:27.009586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.984253+0000","last_change":"2023-10-16T08:16:22.959029+0000","last_active":"2023-10-16T08:16:26.984253+0000","last_peered":"2023-10-16T08:16:26.984253+0000","last_clean":"2023-10-16T08:16:26.984253+0000","last_became_active":"2023-10-16T08:16:22.958713+0000","last_became_peered":"2023-10-16T08:16:22.958713+0000","last_unstale":"2023-10-16T08:16:26.984253+0000","last_undegraded":"2023-10-16T08:16:26.984253+0000","last_fullsized":"2023-10-16T08:16:26.984253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:47.145986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.990281+0000","last_change":"2023-10-16T08:16:22.973255+0000","last_active":"2023-10-16T08:16:26.990281+0000","last_peered":"2023-10-16T08:16:26.990281+0000","last_clean":"2023-10-16T08:16:26.990281+0000","last_became_active":"2023-10-16T08:16:22.972582+0000","last_became_peered":"2023-10-16T08:16:22.972582+0000","last_unstale":"2023-10-16T08:16:26.990281+0000","last_undegraded":"2023-10-16T08:16:26.990281+0000","last_fullsized":"2023-10-16T08:16:26.990281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.004704+0000","last_change":"2023-10-16T08:16:22.969190+0000","last_active":"2023-10-16T08:16:27.004704+0000","last_peered":"2023-10-16T08:16:27.004704+0000","last_clean":"2023-10-16T08:16:27.004704+0000","last_became_active":"2023-10-16T08:16:22.968880+0000","last_became_peered":"2023-10-16T08:16:22.968880+0000","last_unstale":"2023-10-16T08:16:27.004704+0000","last_undegraded":"2023-10-16T08:16:27.004704+0000","last_fullsized":"2023-10-16T08:16:27.004704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:59.314484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987745+0000","last_change":"2023-10-16T08:16:22.975463+0000","last_active":"2023-10-16T08:16:26.987745+0000","last_peered":"2023-10-16T08:16:26.987745+0000","last_clean":"2023-10-16T08:16:26.987745+0000","last_became_active":"2023-10-16T08:16:22.975080+0000","last_became_peered":"2023-10-16T08:16:22.975080+0000","last_unstale":"2023-10-16T08:16:26.987745+0000","last_undegraded":"2023-10-16T08:16:26.987745+0000","last_fullsized":"2023-10-16T08:16:26.987745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:35.692525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987244+0000","last_change":"2023-10-16T08:16:22.975707+0000","last_active":"2023-10-16T08:16:26.987244+0000","last_peered":"2023-10-16T08:16:26.987244+0000","last_clean":"2023-10-16T08:16:26.987244+0000","last_became_active":"2023-10-16T08:16:22.975429+0000","last_became_peered":"2023-10-16T08:16:22.975429+0000","last_unstale":"2023-10-16T08:16:26.987244+0000","last_undegraded":"2023-10-16T08:16:26.987244+0000","last_fullsized":"2023-10-16T08:16:26.987244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:33.039237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.179066+0000","last_change":"2023-10-16T08:16:22.977921+0000","last_active":"2023-10-16T08:16:27.179066+0000","last_peered":"2023-10-16T08:16:27.179066+0000","last_clean":"2023-10-16T08:16:27.179066+0000","last_became_active":"2023-10-16T08:16:22.974938+0000","last_became_peered":"2023-10-16T08:16:22.974938+0000","last_unstale":"2023-10-16T08:16:27.179066+0000","last_undegraded":"2023-10-16T08:16:27.179066+0000","last_fullsized":"2023-10-16T08:16:27.179066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:46:35.197545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.986251+0000","last_change":"2023-10-16T08:16:22.972026+0000","last_active":"2023-10-16T08:16:26.986251+0000","last_peered":"2023-10-16T08:16:26.986251+0000","last_clean":"2023-10-16T08:16:26.986251+0000","last_became_active":"2023-10-16T08:16:22.971655+0000","last_became_peered":"2023-10-16T08:16:22.971655+0000","last_unstale":"2023-10-16T08:16:26.986251+0000","last_undegraded":"2023-10-16T08:16:26.986251+0000","last_fullsized":"2023-10-16T08:16:26.986251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:27.148957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178078+0000","last_change":"2023-10-16T08:16:22.973377+0000","last_active":"2023-10-16T08:16:27.178078+0000","last_peered":"2023-10-16T08:16:27.178078+0000","last_clean":"2023-10-16T08:16:27.178078+0000","last_became_active":"2023-10-16T08:16:22.972888+0000","last_became_peered":"2023-10-16T08:16:22.972888+0000","last_unstale":"2023-10-16T08:16:27.178078+0000","last_undegraded":"2023-10-16T08:16:27.178078+0000","last_fullsized":"2023-10-16T08:16:27.178078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:32:45.901796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994140+0000","last_change":"2023-10-16T08:16:22.976999+0000","last_active":"2023-10-16T08:16:26.994140+0000","last_peered":"2023-10-16T08:16:26.994140+0000","last_clean":"2023-10-16T08:16:26.994140+0000","last_became_active":"2023-10-16T08:16:22.974943+0000","last_became_peered":"2023-10-16T08:16:22.974943+0000","last_unstale":"2023-10-16T08:16:26.994140+0000","last_undegraded":"2023-10-16T08:16:26.994140+0000","last_fullsized":"2023-10-16T08:16:26.994140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:08.529527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178298+0000","last_change":"2023-10-16T08:16:22.979562+0000","last_active":"2023-10-16T08:16:27.178298+0000","last_peered":"2023-10-16T08:16:27.178298+0000","last_clean":"2023-10-16T08:16:27.178298+0000","last_became_active":"2023-10-16T08:16:22.975468+0000","last_became_peered":"2023-10-16T08:16:22.975468+0000","last_unstale":"2023-10-16T08:16:27.178298+0000","last_undegraded":"2023-10-16T08:16:27.178298+0000","last_fullsized":"2023-10-16T08:16:27.178298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:12.431910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.994788+0000","last_change":"2023-10-16T08:16:22.991415+0000","last_active":"2023-10-16T08:16:26.994788+0000","last_peered":"2023-10-16T08:16:26.994788+0000","last_clean":"2023-10-16T08:16:26.994788+0000","last_became_active":"2023-10-16T08:16:22.990410+0000","last_became_peered":"2023-10-16T08:16:22.990410+0000","last_unstale":"2023-10-16T08:16:26.994788+0000","last_undegraded":"2023-10-16T08:16:26.994788+0000","last_fullsized":"2023-10-16T08:16:26.994788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:49.826517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:26.987345+0000","last_change":"2023-10-16T08:16:22.957927+0000","last_active":"2023-10-16T08:16:26.987345+0000","last_peered":"2023-10-16T08:16:26.987345+0000","last_clean":"2023-10-16T08:16:26.987345+0000","last_became_active":"2023-10-16T08:16:22.956909+0000","last_became_peered":"2023-10-16T08:16:22.956909+0000","last_unstale":"2023-10-16T08:16:26.987345+0000","last_undegraded":"2023-10-16T08:16:26.987345+0000","last_fullsized":"2023-10-16T08:16:26.987345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:19:16.906152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.014243+0000","last_change":"2023-10-16T08:16:22.963419+0000","last_active":"2023-10-16T08:16:27.014243+0000","last_peered":"2023-10-16T08:16:27.014243+0000","last_clean":"2023-10-16T08:16:27.014243+0000","last_became_active":"2023-10-16T08:16:22.962692+0000","last_became_peered":"2023-10-16T08:16:22.962692+0000","last_unstale":"2023-10-16T08:16:27.014243+0000","last_undegraded":"2023-10-16T08:16:27.014243+0000","last_fullsized":"2023-10-16T08:16:27.014243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:13.716217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-16T08:16:27.178991+0000","last_change":"2023-10-16T08:16:22.976621+0000","last_active":"2023-10-16T08:16:27.178991+0000","last_peered":"2023-10-16T08:16:27.178991+0000","last_clean":"2023-10-16T08:16:27.178991+0000","last_became_active":"2023-10-16T08:16:22.975391+0000","last_became_peered":"2023-10-16T08:16:22.975391+0000","last_unstale":"2023-10-16T08:16:27.178991+0000","last_undegraded":"2023-10-16T08:16:27.178991+0000","last_fullsized":"2023-10-16T08:16:27.178991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:21.916749+0000","last_clean_scrub_stamp":"2023-10-16T08:16:21.916749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:07.104289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.706}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:16:53.350 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T08:16:53.351 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-16T08:16:53.351 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-16T08:16:53.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-16T08:16:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b09cc9640 1 Processor -- start 2023-10-16T08:16:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b09cc9640 1 -- start start 2023-10-16T08:16:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b09cc9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5b0415ca70 0x7f5b040d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b09cc9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040c65f0 con 0x7f5b040cf7a0 2023-10-16T08:16:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b09cc9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040cdf80 con 0x7f5b040d3550 2023-10-16T08:16:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b09cc9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b040ce820 con 0x7f5b0415ca70 2023-10-16T08:16:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b03fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b040cf7a0 0x7f5b040c6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35828/0 (socket says 172.21.15.32:35828) 2023-10-16T08:16:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b03fff640 1 -- 172.21.15.32:0/2898960851 learned_addr learned my addr 172.21.15.32:0/2898960851 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b037fe640 1 --2- 172.21.15.32:0/2898960851 >> v2:172.21.15.186:3300/0 conn(0x7f5b0415ca70 0x7f5b040d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233948757 0 0) 0x7f5b040c65f0 con 0x7f5b040cf7a0 2023-10-16T08:16:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aec003530 con 0x7f5b040cf7a0 2023-10-16T08:16:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823918793 0 0) 0x7f5aec003530 con 0x7f5b040cf7a0 2023-10-16T08:16:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 >> v2:172.21.15.186:3300/0 conn(0x7f5b0415ca70 msgr2=0x7f5b040d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 --2- 172.21.15.32:0/2898960851 >> v2:172.21.15.186:3300/0 conn(0x7f5b0415ca70 0x7f5b040d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 >> v1:172.21.15.186:6789/0 conn(0x7f5b040d3550 legacy=0x7f5b040cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040ce260 con 0x7f5b040cf7a0 2023-10-16T08:16:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f5afc001a00 con 0x7f5b040cf7a0 2023-10-16T08:16:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 >> v1:172.21.15.32:6789/0 conn(0x7f5b040cf7a0 legacy=0x7f5b040c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5aec003980 0x7f5aec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec0047d0 0x7f5aec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aec005190 0x7f5aec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040ce820 con 0x7f5aec005190 2023-10-16T08:16:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec00c030 con 0x7f5aec0047d0 2023-10-16T08:16:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec00c1f0 con 0x7f5aec003980 2023-10-16T08:16:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b03fff640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aec005190 0x7f5aec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b037fe640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5aec003980 0x7f5aec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.526+0000 7f5b02ffd640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec0047d0 0x7f5aec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 -- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5aec003980 msgr2=0x7f5aec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5aec003980 0x7f5aec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 -- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec0047d0 msgr2=0x7f5aec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec0047d0 0x7f5aec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 -- 172.21.15.32:0/2898960851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec003530 con 0x7f5aec005190 2023-10-16T08:16:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 -- 172.21.15.32:0/2898960851 reap_dead start 2023-10-16T08:16:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b03fff640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aec005190 0x7f5aec00b990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00a800 tx=0x7f5afc01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e710b5fe584cad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0072d0 con 0x7f5aec005190 2023-10-16T08:16:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc0a8070 con 0x7f5aec005190 2023-10-16T08:16:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b027fc640 1 -- 172.21.15.32:0/2898960851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007670 con 0x7f5aec005190 2023-10-16T08:16:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 -- 172.21.15.32:0/2898960851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aec005190 msgr2=0x7f5aec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aec005190 0x7f5aec00b990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00a800 tx=0x7f5afc01d300 comp rx=0 tx=0).stop 2023-10-16T08:16:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 -- 172.21.15.32:0/2898960851 shutdown_connections 2023-10-16T08:16:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/2898960851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aec005190 0x7f5aec00b990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 -- 172.21.15.32:0/2898960851 >> 172.21.15.32:0/2898960851 conn(0x7f5b0409bc50 msgr2=0x7f5b040d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 -- 172.21.15.32:0/2898960851 shutdown_connections 2023-10-16T08:16:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.530+0000 7f5b09cc9640 1 -- 172.21.15.32:0/2898960851 wait complete. 2023-10-16T08:16:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 Processor -- start 2023-10-16T08:16:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 -- start start 2023-10-16T08:16:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec005190 0x7f5b04154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 0x7f5b0415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b040b6340 0x7f5b040b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0415e830 con 0x7f5b0415a000 2023-10-16T08:16:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0415a9a0 con 0x7f5aec005190 2023-10-16T08:16:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b09cc9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040b6d00 con 0x7f5b040b6340 2023-10-16T08:16:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 0x7f5b0415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 0x7f5b0415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45848/0 (socket says 172.21.15.32:45848) 2023-10-16T08:16:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 -- 172.21.15.32:0/3877926270 learned_addr learned my addr 172.21.15.32:0/3877926270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b03fff640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b040b6340 0x7f5b040b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b037fe640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec005190 0x7f5b04154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 -- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b040b6340 msgr2=0x7f5b040b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b040b6340 0x7f5b040b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 -- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec005190 msgr2=0x7f5b04154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec005190 0x7f5b04154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 -- 172.21.15.32:0/3877926270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b041549d0 con 0x7f5b0415a000 2023-10-16T08:16:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b037fe640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec005190 0x7f5b04154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5b02ffd640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 0x7f5b0415a460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5af8007c60 tx=0x7f5af800fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.534+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af80052e0 con 0x7f5b0415a000 2023-10-16T08:16:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af8005440 con 0x7f5b0415a000 2023-10-16T08:16:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8003820 con 0x7f5b0415a000 2023-10-16T08:16:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b04154cb0 con 0x7f5b0415a000 2023-10-16T08:16:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b04155270 con 0x7f5b0415a000 2023-10-16T08:16:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b04074010 con 0x7f5b0415a000 2023-10-16T08:16:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5af8003980 con 0x7f5b0415a000 2023-10-16T08:16:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5aebfff640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5af00664a0 0x7f5af0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5af8083eb0 con 0x7f5b0415a000 2023-10-16T08:16:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5b037fe640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5af00664a0 0x7f5af0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.538+0000 7f5b037fe640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5af00664a0 0x7f5af0068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009b40 tx=0x7f5af4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.542+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5af80510d0 con 0x7f5b0415a000 2023-10-16T08:16:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.734+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5b0415eaa0 con 0x7f5b0415a000 2023-10-16T08:16:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.734+0000 7f5aebfff640 1 -- 172.21.15.32:0/3877926270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5af80546f0 con 0x7f5b0415a000 2023-10-16T08:16:53.741 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:53.741 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-16T08:16:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5af00664a0 msgr2=0x7f5af0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5af00664a0 0x7f5af0068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009b40 tx=0x7f5af4008040 comp rx=0 tx=0).stop 2023-10-16T08:16:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 msgr2=0x7f5b0415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 0x7f5b0415a460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5af8007c60 tx=0x7f5af800fa50 comp rx=0 tx=0).stop 2023-10-16T08:16:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 shutdown_connections 2023-10-16T08:16:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5af00664a0 0x7f5af0068960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b040b6340 0x7f5b040b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0415a000 0x7f5b0415a460 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 --2- 172.21.15.32:0/3877926270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5aec005190 0x7f5b04154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 >> 172.21.15.32:0/3877926270 conn(0x7f5b0409bc50 msgr2=0x7f5b0406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 shutdown_connections 2023-10-16T08:16:53.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.738+0000 7f5b09cc9640 1 -- 172.21.15.32:0/3877926270 wait complete. 2023-10-16T08:16:53.762 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-16T08:16:53.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:53.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.898+0000 7f9fdb046640 1 Processor -- start 2023-10-16T08:16:53.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.898+0000 7f9fdb046640 1 -- start start 2023-10-16T08:16:53.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fdb046640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9fd415cb30 0x7f9fd41608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fdb046640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fd4160e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fdb046640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd40ccbd0 con 0x7f9fd415cf10 2023-10-16T08:16:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fdb046640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd406b180 con 0x7f9fd415cb30 2023-10-16T08:16:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fdb046640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd406b460 con 0x7f9fd4065a80 2023-10-16T08:16:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fd415cf10 0x7f9fd4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35844/0 (socket says 172.21.15.32:35844) 2023-10-16T08:16:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1695820362 learned_addr learned my addr 172.21.15.32:0/1695820362 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:53.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd8de2640 1 --2- 172.21.15.32:0/1695820362 >> v2:172.21.15.186:3300/0 conn(0x7f9fd415cb30 0x7f9fd41608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd3fff640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fd4160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374747297 0 0) 0x7f9fd40ccbd0 con 0x7f9fd415cf10 2023-10-16T08:16:53.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc4003530 con 0x7f9fd415cf10 2023-10-16T08:16:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123146233 0 0) 0x7f9fc4003530 con 0x7f9fd415cf10 2023-10-16T08:16:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 msgr2=0x7f9fd4160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fd4160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 >> v2:172.21.15.186:3300/0 conn(0x7f9fd415cb30 msgr2=0x7f9fd41608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 --2- 172.21.15.32:0/1695820362 >> v2:172.21.15.186:3300/0 conn(0x7f9fd415cb30 0x7f9fd41608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd406c6e0 con 0x7f9fd415cf10 2023-10-16T08:16:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f9fcc001a00 con 0x7f9fd415cf10 2023-10-16T08:16:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd3fff640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fd4160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 >> v1:172.21.15.32:6789/0 conn(0x7f9fd415cf10 legacy=0x7f9fd4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc4003980 0x7f9fc4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fc40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc4004d10 0x7f9fc4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd406b180 con 0x7f9fc4004d10 2023-10-16T08:16:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd406b460 con 0x7f9fc4003980 2023-10-16T08:16:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc400b990 con 0x7f9fd4065a80 2023-10-16T08:16:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd8de2640 1 --2- 172.21.15.32:0/1695820362 >> v2:172.21.15.186:3300/0 conn(0x7f9fd415cb30 0x7f9fd41608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc4004d10 0x7f9fc4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd3fff640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fc40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd8de2640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc4003980 0x7f9fc4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 msgr2=0x7f9fc40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fd4065a80 0x7f9fc40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc4003980 msgr2=0x7f9fc4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc4003980 0x7f9fc4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.902+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1695820362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0009420 con 0x7f9fc4004d10 2023-10-16T08:16:53.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc4004d10 0x7f9fc4005190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00a860 tx=0x7f9fcc01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c272e2687f1443a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0aa6f0 con 0x7f9fc4004d10 2023-10-16T08:16:53.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0aa850 con 0x7f9fc4004d10 2023-10-16T08:16:53.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fd37fe640 1 -- 172.21.15.32:0/1695820362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0aac20 con 0x7f9fc4004d10 2023-10-16T08:16:53.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fdb046640 1 -- 172.21.15.32:0/1695820362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc4004d10 msgr2=0x7f9fc4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1695820362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc4004d10 0x7f9fc4005190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00a860 tx=0x7f9fcc01fac0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1695820362 reap_dead start 2023-10-16T08:16:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fdb046640 1 -- 172.21.15.32:0/1695820362 shutdown_connections 2023-10-16T08:16:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fdb046640 1 -- 172.21.15.32:0/1695820362 >> 172.21.15.32:0/1695820362 conn(0x7f9fd40b0880 msgr2=0x7f9fd40b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fdb046640 1 -- 172.21.15.32:0/1695820362 shutdown_connections 2023-10-16T08:16:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.906+0000 7f9fdb046640 1 -- 172.21.15.32:0/1695820362 wait complete. 2023-10-16T08:16:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 Processor -- start 2023-10-16T08:16:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- start start 2023-10-16T08:16:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc4004d10 0x7f9fd415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fd41f57a0 0x7f9fd41f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 0x7f9fd40c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd415bb30 con 0x7f9fd40c3c50 2023-10-16T08:16:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40c3800 con 0x7f9fd41f57a0 2023-10-16T08:16:53.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41f6160 con 0x7f9fc4004d10 2023-10-16T08:16:53.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 0x7f9fd40c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 0x7f9fd40c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45866/0 (socket says 172.21.15.32:45866) 2023-10-16T08:16:53.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1604367673 learned_addr learned my addr 172.21.15.32:0/1604367673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:53.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd8de2640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc4004d10 0x7f9fd415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd3fff640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fd41f57a0 0x7f9fd41f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc4004d10 msgr2=0x7f9fd415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc4004d10 0x7f9fd415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fd41f57a0 msgr2=0x7f9fd41f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fd41f57a0 0x7f9fd41f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 -- 172.21.15.32:0/1604367673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd415df50 con 0x7f9fd40c3c50 2023-10-16T08:16:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd95e3640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 0x7f9fd40c40d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc01f8f0 tx=0x7f9fc4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0aa6f0 con 0x7f9fd40c3c50 2023-10-16T08:16:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc001fe0 con 0x7f9fd40c3c50 2023-10-16T08:16:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0019c0 con 0x7f9fd40c3c50 2023-10-16T08:16:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd415bfe0 con 0x7f9fd40c3c50 2023-10-16T08:16:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd415e480 con 0x7f9fd40c3c50 2023-10-16T08:16:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.910+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd40b3a80 con 0x7f9fd40c3c50 2023-10-16T08:16:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.914+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9fcc0bc070 con 0x7f9fd40c3c50 2023-10-16T08:16:53.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.914+0000 7f9fd17fa640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9fb80664a0 0x7f9fb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:53.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.914+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9fcc121800 con 0x7f9fd40c3c50 2023-10-16T08:16:53.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.914+0000 7f9fd8de2640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9fb80664a0 0x7f9fb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:53.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.914+0000 7f9fd8de2640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9fb80664a0 0x7f9fb8068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80093f0 tx=0x7f9fc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:53.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:53.918+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9fcc0ee9a0 con 0x7f9fd40c3c50 2023-10-16T08:16:54.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.078+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd4074010 con 0x7f9fd40c3c50 2023-10-16T08:16:54.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.078+0000 7f9fd17fa640 1 -- 172.21.15.32:0/1604367673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f9fcc0f1fc0 con 0x7f9fd40c3c50 2023-10-16T08:16:54.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:54.085 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9fb80664a0 msgr2=0x7f9fb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9fb80664a0 0x7f9fb8068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80093f0 tx=0x7f9fc8008040 comp rx=0 tx=0).stop 2023-10-16T08:16:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 msgr2=0x7f9fd40c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 0x7f9fd40c40d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc01f8f0 tx=0x7f9fc4005800 comp rx=0 tx=0).stop 2023-10-16T08:16:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 shutdown_connections 2023-10-16T08:16:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd40c3c50 0x7f9fd40c40d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fd41f57a0 0x7f9fd41f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9fb80664a0 0x7f9fb8068960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 --2- 172.21.15.32:0/1604367673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9fc4004d10 0x7f9fd415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 >> 172.21.15.32:0/1604367673 conn(0x7f9fd40b0880 msgr2=0x7f9fd409ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 shutdown_connections 2023-10-16T08:16:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.082+0000 7f9fdb046640 1 -- 172.21.15.32:0/1604367673 wait complete. 2023-10-16T08:16:54.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:16:54.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:16:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.246+0000 7ff0c6aad640 1 Processor -- start 2023-10-16T08:16:54.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.246+0000 7ff0c6aad640 1 -- start start 2023-10-16T08:16:54.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c6aad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0c00cf210 0x7ff0c00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c6aad640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00cfb30 con 0x7ff0c015cdc0 2023-10-16T08:16:54.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c6aad640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00d0880 con 0x7ff0c00d3320 2023-10-16T08:16:54.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c6aad640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00b1c00 con 0x7ff0c00cf210 2023-10-16T08:16:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c5aab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0c00cf210 0x7ff0c00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c52aa640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0c015cdc0 0x7ff0c00b2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35854/0 (socket says 172.21.15.32:35854) 2023-10-16T08:16:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c52aa640 1 -- 172.21.15.32:0/196740108 learned_addr learned my addr 172.21.15.32:0/196740108 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c5aab640 1 -- 172.21.15.32:0/196740108 >> v1:172.21.15.186:6789/0 conn(0x7ff0c00d3320 legacy=0x7ff0c00b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c5aab640 1 -- 172.21.15.32:0/196740108 >> v1:172.21.15.32:6789/0 conn(0x7ff0c015cdc0 legacy=0x7ff0c00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c5aab640 1 -- 172.21.15.32:0/196740108 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00b1ee0 con 0x7ff0c00cf210 2023-10-16T08:16:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.250+0000 7ff0c4aa9640 1 -- 172.21.15.32:0/196740108 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156707441 0 0) 0x7ff0c00cfb30 con 0x7ff0c015cdc0 2023-10-16T08:16:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c5aab640 1 --2- 172.21.15.32:0/196740108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c00cf5f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0097a0 tx=0x7ff0bc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb6982fc6ab5daa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c4aa9640 1 -- 172.21.15.32:0/196740108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ac600 con 0x7ff0c00cf210 2023-10-16T08:16:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c4aa9640 1 -- 172.21.15.32:0/196740108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a8050 con 0x7ff0c00cf210 2023-10-16T08:16:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c4aa9640 1 -- 172.21.15.32:0/196740108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ac980 con 0x7ff0c00cf210 2023-10-16T08:16:54.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 -- 172.21.15.32:0/196740108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 msgr2=0x7ff0c00cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/196740108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c00cf5f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0097a0 tx=0x7ff0bc01c3a0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 -- 172.21.15.32:0/196740108 shutdown_connections 2023-10-16T08:16:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/196740108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c00cf5f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 -- 172.21.15.32:0/196740108 >> 172.21.15.32:0/196740108 conn(0x7ff0c00d2800 msgr2=0x7ff0c00d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 -- 172.21.15.32:0/196740108 shutdown_connections 2023-10-16T08:16:54.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 -- 172.21.15.32:0/196740108 wait complete. 2023-10-16T08:16:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 Processor -- start 2023-10-16T08:16:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.254+0000 7ff0c6aad640 1 -- start start 2023-10-16T08:16:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c0153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0c00d3320 0x7ff0c0156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0c015cdc0 0x7ff0c0150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0159620 con 0x7ff0c00cf210 2023-10-16T08:16:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c015a1a0 con 0x7ff0c015cdc0 2023-10-16T08:16:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0157870 con 0x7ff0c00d3320 2023-10-16T08:16:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c0153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c0153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45884/0 (socket says 172.21.15.32:45884) 2023-10-16T08:16:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 -- 172.21.15.32:0/4162725933 learned_addr learned my addr 172.21.15.32:0/4162725933 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c62ac640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0c015cdc0 0x7ff0c0150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c52aa640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0c00d3320 0x7ff0c0156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 -- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0c00d3320 msgr2=0x7ff0c0156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0c00d3320 0x7ff0c0156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 -- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0c015cdc0 msgr2=0x7ff0c0150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0c015cdc0 0x7ff0c0150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 -- 172.21.15.32:0/4162725933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0093b0 con 0x7ff0c00cf210 2023-10-16T08:16:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c62ac640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0c015cdc0 0x7ff0c0150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c52aa640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0c00d3320 0x7ff0c0156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c5aab640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c0153b40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc018fb0 tx=0x7ff0bc01abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0b7650 con 0x7ff0c00cf210 2023-10-16T08:16:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0ace90 con 0x7ff0c00cf210 2023-10-16T08:16:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c0159870 con 0x7ff0c00cf210 2023-10-16T08:16:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0b79f0 con 0x7ff0c00cf210 2023-10-16T08:16:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c0159e30 con 0x7ff0c00cf210 2023-10-16T08:16:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c004e9c0 con 0x7ff0c00cf210 2023-10-16T08:16:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.258+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff0bc0b3070 con 0x7ff0c00cf210 2023-10-16T08:16:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.262+0000 7ff0b67fc640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0a40664a0 0x7ff0a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.262+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0bc11e900 con 0x7ff0c00cf210 2023-10-16T08:16:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.262+0000 7ff0c52aa640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0a40664a0 0x7ff0a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.262+0000 7ff0c52aa640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0a40664a0 0x7ff0a4068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff0c006baf0 tx=0x7ff0b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.266+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0bc0ebba0 con 0x7ff0c00cf210 2023-10-16T08:16:54.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.422+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0c00b8690 con 0x7ff0c00cf210 2023-10-16T08:16:54.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.422+0000 7ff0b67fc640 1 -- 172.21.15.32:0/4162725933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0bc0b1020 con 0x7ff0c00cf210 2023-10-16T08:16:54.428 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:16:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0a40664a0 msgr2=0x7ff0a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0a40664a0 0x7ff0a4068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff0c006baf0 tx=0x7ff0b0009040 comp rx=0 tx=0).stop 2023-10-16T08:16:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 msgr2=0x7ff0c0153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c0153b40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc018fb0 tx=0x7ff0bc01abe0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 shutdown_connections 2023-10-16T08:16:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0c015cdc0 0x7ff0c0150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0a40664a0 0x7ff0a4068960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0c00d3320 0x7ff0c0156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 --2- 172.21.15.32:0/4162725933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c00cf210 0x7ff0c0153b40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 >> 172.21.15.32:0/4162725933 conn(0x7ff0c00d2800 msgr2=0x7ff0c00d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 shutdown_connections 2023-10-16T08:16:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.426+0000 7ff0c6aad640 1 -- 172.21.15.32:0/4162725933 wait complete. 2023-10-16T08:16:54.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:16:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 Processor -- start 2023-10-16T08:16:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 -- start start 2023-10-16T08:16:54.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f388c069e90 0x7f388c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f388c06a270 0x7f388c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c15ae30 con 0x7f388c0702f0 2023-10-16T08:16:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0cb090 con 0x7f388c06a270 2023-10-16T08:16:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f38939f8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0ca510 con 0x7f388c069e90 2023-10-16T08:16:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f388c06a270 0x7f388c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f388c06a270 0x7f388c0c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45890/0 (socket says 172.21.15.32:45890) 2023-10-16T08:16:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 -- 172.21.15.32:0/4103461788 learned_addr learned my addr 172.21.15.32:0/4103461788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3891794640 1 --2- 172.21.15.32:0/4103461788 >> v2:172.21.15.186:3300/0 conn(0x7f388c069e90 0x7f388c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 -- 172.21.15.32:0/4103461788 >> v2:172.21.15.186:3300/0 conn(0x7f388c069e90 msgr2=0x7f388c06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 --2- 172.21.15.32:0/4103461788 >> v2:172.21.15.186:3300/0 conn(0x7f388c069e90 0x7f388c06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 -- 172.21.15.32:0/4103461788 >> v1:172.21.15.186:6789/0 conn(0x7f388c0702f0 legacy=0x7f388c070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.590+0000 7f3890f93640 1 -- 172.21.15.32:0/4103461788 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0ca7f0 con 0x7f388c06a270 2023-10-16T08:16:54.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f3890f93640 1 --2- 172.21.15.32:0/4103461788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c06a270 0x7f388c0c9ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f388c071590 tx=0x7f387c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47c8361c33b9498 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f3883fff640 1 -- 172.21.15.32:0/4103461788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0aa5f0 con 0x7f388c06a270 2023-10-16T08:16:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f3883fff640 1 -- 172.21.15.32:0/4103461788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c0aa750 con 0x7f388c06a270 2023-10-16T08:16:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f3883fff640 1 -- 172.21.15.32:0/4103461788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0aaae0 con 0x7f388c06a270 2023-10-16T08:16:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 -- 172.21.15.32:0/4103461788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c06a270 msgr2=0x7f388c0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 --2- 172.21.15.32:0/4103461788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c06a270 0x7f388c0c9ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f388c071590 tx=0x7f387c01c490 comp rx=0 tx=0).stop 2023-10-16T08:16:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 -- 172.21.15.32:0/4103461788 shutdown_connections 2023-10-16T08:16:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 --2- 172.21.15.32:0/4103461788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c06a270 0x7f388c0c9ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 --2- 172.21.15.32:0/4103461788 >> v2:172.21.15.186:3300/0 conn(0x7f388c069e90 0x7f388c06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 -- 172.21.15.32:0/4103461788 >> 172.21.15.32:0/4103461788 conn(0x7f388c0b0da0 msgr2=0x7f388c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 -- 172.21.15.32:0/4103461788 shutdown_connections 2023-10-16T08:16:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.594+0000 7f38939f8640 1 -- 172.21.15.32:0/4103461788 wait complete. 2023-10-16T08:16:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 Processor -- start 2023-10-16T08:16:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- start start 2023-10-16T08:16:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 0x7f388c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f388c06a270 0x7f388c0af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f388c0702f0 0x7f388c0c1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c154880 con 0x7f388c069e90 2023-10-16T08:16:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0c7d60 con 0x7f388c06a270 2023-10-16T08:16:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0c8040 con 0x7f388c0702f0 2023-10-16T08:16:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 0x7f388c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 0x7f388c152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45902/0 (socket says 172.21.15.32:45902) 2023-10-16T08:16:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 -- 172.21.15.32:0/793459275 learned_addr learned my addr 172.21.15.32:0/793459275 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3890f93640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f388c06a270 0x7f388c0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891f95640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f388c0702f0 0x7f388c0c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 -- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f388c0702f0 msgr2=0x7f388c0c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f388c0702f0 0x7f388c0c1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 -- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f388c06a270 msgr2=0x7f388c0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f388c06a270 0x7f388c0af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 -- 172.21.15.32:0/793459275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0094d0 con 0x7f388c069e90 2023-10-16T08:16:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3891794640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 0x7f388c152810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f388800d800 tx=0x7f388800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800b6c0 con 0x7f388c069e90 2023-10-16T08:16:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800b820 con 0x7f388c069e90 2023-10-16T08:16:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c0c8320 con 0x7f388c069e90 2023-10-16T08:16:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800bb90 con 0x7f388c069e90 2023-10-16T08:16:54.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c0afa70 con 0x7f388c069e90 2023-10-16T08:16:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.598+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c04e170 con 0x7f388c069e90 2023-10-16T08:16:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.602+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f388800bcf0 con 0x7f388c069e90 2023-10-16T08:16:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.602+0000 7f3881ffb640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38700664a0 0x7f3870068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.602+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3888084630 con 0x7f388c069e90 2023-10-16T08:16:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.602+0000 7f3890f93640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38700664a0 0x7f3870068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.602+0000 7f3890f93640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38700664a0 0x7f3870068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f387c0ae000 tx=0x7f387c0b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.606+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f38880517b0 con 0x7f388c069e90 2023-10-16T08:16:54.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.762+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f388c0c2090 con 0x7f388c069e90 2023-10-16T08:16:54.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f3881ffb640 1 -- 172.21.15.32:0/793459275 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3888054dd0 con 0x7f388c069e90 2023-10-16T08:16:54.771 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38700664a0 msgr2=0x7f3870068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38700664a0 0x7f3870068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f387c0ae000 tx=0x7f387c0b5040 comp rx=0 tx=0).stop 2023-10-16T08:16:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 msgr2=0x7f388c152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 0x7f388c152810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f388800d800 tx=0x7f388800dcc0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 shutdown_connections 2023-10-16T08:16:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38700664a0 0x7f3870068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f388c0702f0 0x7f388c0c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f388c06a270 0x7f388c0af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 --2- 172.21.15.32:0/793459275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f388c069e90 0x7f388c152810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 >> 172.21.15.32:0/793459275 conn(0x7f388c0b0da0 msgr2=0x7f388c072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 shutdown_connections 2023-10-16T08:16:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.766+0000 7f38939f8640 1 -- 172.21.15.32:0/793459275 wait complete. 2023-10-16T08:16:54.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:16:54.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.930+0000 7fd34cf65640 1 Processor -- start 2023-10-16T08:16:54.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.930+0000 7fd34cf65640 1 -- start start 2023-10-16T08:16:54.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34cf65640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 0x7fd34806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34cf65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 0x7fd3480c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34cf65640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3480b4390 con 0x7fd3480b3f00 2023-10-16T08:16:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34cf65640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3480cd740 con 0x7fd348159900 2023-10-16T08:16:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34cf65640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480b4600 con 0x7fd34806a5a0 2023-10-16T08:16:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd346d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 0x7fd3480c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 0x7fd34806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 0x7fd34806e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:34726/0 (socket says 172.21.15.32:34726) 2023-10-16T08:16:54.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34659c640 1 -- 172.21.15.32:0/82878110 learned_addr learned my addr 172.21.15.32:0/82878110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:54.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd346d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 0x7fd3480c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:42376/0 (socket says 172.21.15.32:42376) 2023-10-16T08:16:54.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34559a640 1 -- 172.21.15.32:0/82878110 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564507968 0 0) 0x7fd3480b4390 con 0x7fd3480b3f00 2023-10-16T08:16:54.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34559a640 1 -- 172.21.15.32:0/82878110 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd330005740 con 0x7fd3480b3f00 2023-10-16T08:16:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd346d9d640 1 -- 172.21.15.32:0/82878110 >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 msgr2=0x7fd34806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd346d9d640 1 --2- 172.21.15.32:0/82878110 >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 0x7fd34806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd346d9d640 1 -- 172.21.15.32:0/82878110 >> v1:172.21.15.186:6789/0 conn(0x7fd3480b3f00 legacy=0x7fd348069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd346d9d640 1 -- 172.21.15.32:0/82878110 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480c8dd0 con 0x7fd34806a5a0 2023-10-16T08:16:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.934+0000 7fd34659c640 1 --2- 172.21.15.32:0/82878110 >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 0x7fd34806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd346d9d640 1 --2- 172.21.15.32:0/82878110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 0x7fd3480c9b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd340009d90 tx=0x7fd34001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1601b3b33b73249d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34559a640 1 -- 172.21.15.32:0/82878110 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400a59e0 con 0x7fd34806a5a0 2023-10-16T08:16:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34559a640 1 -- 172.21.15.32:0/82878110 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3400a5b40 con 0x7fd34806a5a0 2023-10-16T08:16:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34559a640 1 -- 172.21.15.32:0/82878110 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400ad440 con 0x7fd34806a5a0 2023-10-16T08:16:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 -- 172.21.15.32:0/82878110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 msgr2=0x7fd3480c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 --2- 172.21.15.32:0/82878110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 0x7fd3480c9b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd340009d90 tx=0x7fd34001c920 comp rx=0 tx=0).stop 2023-10-16T08:16:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 -- 172.21.15.32:0/82878110 shutdown_connections 2023-10-16T08:16:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 --2- 172.21.15.32:0/82878110 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd34806a5a0 0x7fd3480c9b90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 --2- 172.21.15.32:0/82878110 >> v2:172.21.15.186:3300/0 conn(0x7fd348159900 0x7fd34806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 -- 172.21.15.32:0/82878110 >> 172.21.15.32:0/82878110 conn(0x7fd3480b0da0 msgr2=0x7fd3480ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 -- 172.21.15.32:0/82878110 shutdown_connections 2023-10-16T08:16:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 -- 172.21.15.32:0/82878110 wait complete. 2023-10-16T08:16:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 Processor -- start 2023-10-16T08:16:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.938+0000 7fd34cf65640 1 -- start start 2023-10-16T08:16:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 0x7fd3480cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3480b3f00 0x7fd3480cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd348159900 0x7fd3480c2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480c6bd0 con 0x7fd34806a5a0 2023-10-16T08:16:54.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480c5fc0 con 0x7fd348159900 2023-10-16T08:16:54.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480c62a0 con 0x7fd3480b3f00 2023-10-16T08:16:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 0x7fd3480cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 0x7fd3480cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45904/0 (socket says 172.21.15.32:45904) 2023-10-16T08:16:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 -- 172.21.15.32:0/497011036 learned_addr learned my addr 172.21.15.32:0/497011036 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd345d9b640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3480b3f00 0x7fd3480cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd346d9d640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd348159900 0x7fd3480c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 -- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3480b3f00 msgr2=0x7fd3480cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3480b3f00 0x7fd3480cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 -- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd348159900 msgr2=0x7fd3480c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd348159900 0x7fd3480c2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 -- 172.21.15.32:0/497011036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380094d0 con 0x7fd34806a5a0 2023-10-16T08:16:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34659c640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 0x7fd3480cc300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd33800e930 tx=0x7fd33800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800ca00 con 0x7fd34806a5a0 2023-10-16T08:16:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800cb60 con 0x7fd34806a5a0 2023-10-16T08:16:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338013470 con 0x7fd34806a5a0 2023-10-16T08:16:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd340009a70 con 0x7fd34806a5a0 2023-10-16T08:16:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3480c6860 con 0x7fd34806a5a0 2023-10-16T08:16:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.942+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3480c9650 con 0x7fd34806a5a0 2023-10-16T08:16:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.946+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd3380135d0 con 0x7fd34806a5a0 2023-10-16T08:16:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.946+0000 7fd336ffd640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3280664a0 0x7fd328068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.946+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd338084360 con 0x7fd34806a5a0 2023-10-16T08:16:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.946+0000 7fd345d9b640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3280664a0 0x7fd328068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.946+0000 7fd345d9b640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3280664a0 0x7fd328068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd3480c5db0 tx=0x7fd33c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:54.950+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd3480c9650 con 0x7fd34806a5a0 2023-10-16T08:16:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.110+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3480c9650 con 0x7fd34806a5a0 2023-10-16T08:16:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.110+0000 7fd336ffd640 1 -- 172.21.15.32:0/497011036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3480c9650 con 0x7fd34806a5a0 2023-10-16T08:16:55.117 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:16:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3280664a0 msgr2=0x7fd328068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3280664a0 0x7fd328068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd3480c5db0 tx=0x7fd33c009040 comp rx=0 tx=0).stop 2023-10-16T08:16:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 msgr2=0x7fd3480cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 0x7fd3480cc300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd33800e930 tx=0x7fd33800edf0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 shutdown_connections 2023-10-16T08:16:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd348159900 0x7fd3480c2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3280664a0 0x7fd328068960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3480b3f00 0x7fd3480cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 --2- 172.21.15.32:0/497011036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34806a5a0 0x7fd3480cc300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 >> 172.21.15.32:0/497011036 conn(0x7fd3480b0da0 msgr2=0x7fd3480afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 shutdown_connections 2023-10-16T08:16:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.114+0000 7fd34cf65640 1 -- 172.21.15.32:0/497011036 wait complete. 2023-10-16T08:16:55.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:16:55.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 Processor -- start 2023-10-16T08:16:55.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 -- start start 2023-10-16T08:16:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fda24069e90 0x7fda2406d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda2406a270 0x7fda240c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2415ae30 con 0x7fda240702f0 2023-10-16T08:16:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda240cb090 con 0x7fda24069e90 2023-10-16T08:16:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda28c91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda240ca510 con 0x7fda2406a270 2023-10-16T08:16:55.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda22d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fda240702f0 0x7fda24070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35870/0 (socket says 172.21.15.32:35870) 2023-10-16T08:16:55.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda22d9d640 1 -- 172.21.15.32:0/506248002 learned_addr learned my addr 172.21.15.32:0/506248002 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda1bfff640 1 --2- 172.21.15.32:0/506248002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda2406a270 0x7fda240c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda2259c640 1 --2- 172.21.15.32:0/506248002 >> v2:172.21.15.186:3300/0 conn(0x7fda24069e90 0x7fda2406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda21d9b640 1 -- 172.21.15.32:0/506248002 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257556463 0 0) 0x7fda2415ae30 con 0x7fda240702f0 2023-10-16T08:16:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.278+0000 7fda21d9b640 1 -- 172.21.15.32:0/506248002 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda10003530 con 0x7fda240702f0 2023-10-16T08:16:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda1bfff640 1 -- 172.21.15.32:0/506248002 >> v2:172.21.15.186:3300/0 conn(0x7fda24069e90 msgr2=0x7fda2406d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda1bfff640 1 --2- 172.21.15.32:0/506248002 >> v2:172.21.15.186:3300/0 conn(0x7fda24069e90 0x7fda2406d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda1bfff640 1 -- 172.21.15.32:0/506248002 >> v1:172.21.15.32:6789/0 conn(0x7fda240702f0 legacy=0x7fda24070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda1bfff640 1 -- 172.21.15.32:0/506248002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda240ca7f0 con 0x7fda2406a270 2023-10-16T08:16:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda2259c640 1 --2- 172.21.15.32:0/506248002 >> v2:172.21.15.186:3300/0 conn(0x7fda24069e90 0x7fda2406d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda1bfff640 1 --2- 172.21.15.32:0/506248002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda2406a270 0x7fda240c9ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0097c0 tx=0x7fda0c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0a41bc6e9bdd0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda21d9b640 1 -- 172.21.15.32:0/506248002 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0ab650 con 0x7fda2406a270 2023-10-16T08:16:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda21d9b640 1 -- 172.21.15.32:0/506248002 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c0ab7b0 con 0x7fda2406a270 2023-10-16T08:16:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda21d9b640 1 -- 172.21.15.32:0/506248002 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0abb40 con 0x7fda2406a270 2023-10-16T08:16:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda28c91640 1 -- 172.21.15.32:0/506248002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda2406a270 msgr2=0x7fda240c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.282+0000 7fda28c91640 1 --2- 172.21.15.32:0/506248002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda2406a270 0x7fda240c9ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0097c0 tx=0x7fda0c01c450 comp rx=0 tx=0).stop 2023-10-16T08:16:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- 172.21.15.32:0/506248002 shutdown_connections 2023-10-16T08:16:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 --2- 172.21.15.32:0/506248002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda2406a270 0x7fda240c9ce0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 --2- 172.21.15.32:0/506248002 >> v2:172.21.15.186:3300/0 conn(0x7fda24069e90 0x7fda2406d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- 172.21.15.32:0/506248002 >> 172.21.15.32:0/506248002 conn(0x7fda240b0da0 msgr2=0x7fda240ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- 172.21.15.32:0/506248002 shutdown_connections 2023-10-16T08:16:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- 172.21.15.32:0/506248002 wait complete. 2023-10-16T08:16:55.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 Processor -- start 2023-10-16T08:16:55.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- start start 2023-10-16T08:16:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda24069e90 0x7fda24156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2406a270 0x7fda24158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 0x7fda24159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24158030 con 0x7fda240702f0 2023-10-16T08:16:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda24154aa0 con 0x7fda2406a270 2023-10-16T08:16:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda28c91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda24157040 con 0x7fda24069e90 2023-10-16T08:16:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 0x7fda24159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 0x7fda24159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45912/0 (socket says 172.21.15.32:45912) 2023-10-16T08:16:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 -- 172.21.15.32:0/323392108 learned_addr learned my addr 172.21.15.32:0/323392108 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda2259c640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda24069e90 0x7fda24156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda1bfff640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2406a270 0x7fda24158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 -- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda24069e90 msgr2=0x7fda24156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda24069e90 0x7fda24156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 -- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2406a270 msgr2=0x7fda24158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2406a270 0x7fda24158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.286+0000 7fda22d9d640 1 -- 172.21.15.32:0/323392108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14009420 con 0x7fda240702f0 2023-10-16T08:16:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda22d9d640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 0x7fda24159060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00cb40 tx=0x7fda1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00aca0 con 0x7fda240702f0 2023-10-16T08:16:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c00ae00 con 0x7fda240702f0 2023-10-16T08:16:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c012620 con 0x7fda240702f0 2023-10-16T08:16:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c009420 con 0x7fda240702f0 2023-10-16T08:16:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda24157a70 con 0x7fda240702f0 2023-10-16T08:16:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2404e170 con 0x7fda240702f0 2023-10-16T08:16:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda1c007070 con 0x7fda240702f0 2023-10-16T08:16:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda1affd640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fda040664a0 0x7fda04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fda1c084250 con 0x7fda240702f0 2023-10-16T08:16:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.290+0000 7fda2259c640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fda040664a0 0x7fda04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.294+0000 7fda2259c640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fda040664a0 0x7fda04068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fda14009be0 tx=0x7fda14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:55.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.294+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda1c051340 con 0x7fda240702f0 2023-10-16T08:16:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.450+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda240c2090 con 0x7fda240702f0 2023-10-16T08:16:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.450+0000 7fda1affd640 1 -- 172.21.15.32:0/323392108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda1c054960 con 0x7fda240702f0 2023-10-16T08:16:55.457 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fda040664a0 msgr2=0x7fda04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fda040664a0 0x7fda04068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fda14009be0 tx=0x7fda14008040 comp rx=0 tx=0).stop 2023-10-16T08:16:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 msgr2=0x7fda24159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 0x7fda24159060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00cb40 tx=0x7fda1c00a060 comp rx=0 tx=0).stop 2023-10-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 shutdown_connections 2023-10-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda240702f0 0x7fda24159060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda2406a270 0x7fda24158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fda040664a0 0x7fda04068960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 --2- 172.21.15.32:0/323392108 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fda24069e90 0x7fda24156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 >> 172.21.15.32:0/323392108 conn(0x7fda240b0da0 msgr2=0x7fda240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 shutdown_connections 2023-10-16T08:16:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.454+0000 7fda28c91640 1 -- 172.21.15.32:0/323392108 wait complete. 2023-10-16T08:16:55.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:55.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.614+0000 7f39e7fc5640 1 Processor -- start 2023-10-16T08:16:55.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.614+0000 7f39e7fc5640 1 -- start start 2023-10-16T08:16:55.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e7fc5640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39e00b07f0 0x7f39e015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e7fc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e0164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e7fc5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e015be60 con 0x7f39e0070c40 2023-10-16T08:16:55.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e7fc5640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e015b3b0 con 0x7f39e00b07f0 2023-10-16T08:16:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e7fc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e015c1f0 con 0x7f39e015d690 2023-10-16T08:16:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e6562640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e0164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5560640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39e00b07f0 0x7f39e015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5d61640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f39e0070c40 0x7f39e00b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:48676/0 (socket says 172.21.15.32:48676) 2023-10-16T08:16:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5d61640 1 -- 172.21.15.32:0/4001192091 learned_addr learned my addr 172.21.15.32:0/4001192091 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:55.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e4d5f640 1 -- 172.21.15.32:0/4001192091 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198555649 0 0) 0x7f39e015be60 con 0x7f39e0070c40 2023-10-16T08:16:55.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e4d5f640 1 -- 172.21.15.32:0/4001192091 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39cc003530 con 0x7f39e0070c40 2023-10-16T08:16:55.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5560640 1 -- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 msgr2=0x7f39e0164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:55.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5560640 1 --2- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e0164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5560640 1 -- 172.21.15.32:0/4001192091 >> v1:172.21.15.186:6789/0 conn(0x7f39e0070c40 legacy=0x7f39e00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.618+0000 7f39e5560640 1 -- 172.21.15.32:0/4001192091 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0071020 con 0x7f39e00b07f0 2023-10-16T08:16:55.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e5560640 1 --2- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e015ae70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39d00097c0 tx=0x7f39d001c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cf8bf1ae9bc0c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:55.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e4d5f640 1 -- 172.21.15.32:0/4001192091 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00ad660 con 0x7f39e00b07f0 2023-10-16T08:16:55.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e4d5f640 1 -- 172.21.15.32:0/4001192091 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d00ad7c0 con 0x7f39e00b07f0 2023-10-16T08:16:55.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e4d5f640 1 -- 172.21.15.32:0/4001192091 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00adb50 con 0x7f39e00b07f0 2023-10-16T08:16:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 -- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 msgr2=0x7f39e015ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e015ae70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39d00097c0 tx=0x7f39d001c360 comp rx=0 tx=0).stop 2023-10-16T08:16:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 -- 172.21.15.32:0/4001192091 shutdown_connections 2023-10-16T08:16:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e0164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/4001192091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e015ae70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 -- 172.21.15.32:0/4001192091 >> 172.21.15.32:0/4001192091 conn(0x7f39e006c190 msgr2=0x7f39e006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 -- 172.21.15.32:0/4001192091 shutdown_connections 2023-10-16T08:16:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 -- 172.21.15.32:0/4001192091 wait complete. 2023-10-16T08:16:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 Processor -- start 2023-10-16T08:16:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.622+0000 7f39e7fc5640 1 -- start start 2023-10-16T08:16:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 0x7f39e014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e01516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e00a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0151220 con 0x7f39e0070c40 2023-10-16T08:16:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0150330 con 0x7f39e00b07f0 2023-10-16T08:16:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0150610 con 0x7f39e015d690 2023-10-16T08:16:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 0x7f39e014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 0x7f39e014f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45918/0 (socket says 172.21.15.32:45918) 2023-10-16T08:16:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 -- 172.21.15.32:0/3806662085 learned_addr learned my addr 172.21.15.32:0/3806662085 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e6562640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e00a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5560640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e01516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 -- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 msgr2=0x7f39e00a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e00a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 -- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 msgr2=0x7f39e01516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e01516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 -- 172.21.15.32:0/3806662085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d0009420 con 0x7f39e0070c40 2023-10-16T08:16:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e5d61640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 0x7f39e014f040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c420 tx=0x7f39dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc005cf0 con 0x7f39e0070c40 2023-10-16T08:16:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc005e50 con 0x7f39e0070c40 2023-10-16T08:16:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e01508f0 con 0x7f39e0070c40 2023-10-16T08:16:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0072d0 con 0x7f39e0070c40 2023-10-16T08:16:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0150eb0 con 0x7f39e0070c40 2023-10-16T08:16:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0074010 con 0x7f39e0070c40 2023-10-16T08:16:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.626+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39dc007430 con 0x7f39e0070c40 2023-10-16T08:16:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.630+0000 7f39d67fc640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c40664a0 0x7f39c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.630+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39dc0836a0 con 0x7f39e0070c40 2023-10-16T08:16:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.630+0000 7f39e5560640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c40664a0 0x7f39c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.630+0000 7f39e5560640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c40664a0 0x7f39c4068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39d00af000 tx=0x7f39d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.634+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39dc050850 con 0x7f39e0070c40 2023-10-16T08:16:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e00ccc30 con 0x7f39e0070c40 2023-10-16T08:16:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39d67fc640 1 -- 172.21.15.32:0/3806662085 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f39dc053e70 con 0x7f39e0070c40 2023-10-16T08:16:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:55.831 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:16:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:16:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c40664a0 msgr2=0x7f39c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c40664a0 0x7f39c4068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39d00af000 tx=0x7f39d0008040 comp rx=0 tx=0).stop 2023-10-16T08:16:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 msgr2=0x7f39e014f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 0x7f39e014f040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c420 tx=0x7f39dc00c8e0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 shutdown_connections 2023-10-16T08:16:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c40664a0 0x7f39c4068960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e015d690 0x7f39e00a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e00b07f0 0x7f39e01516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 --2- 172.21.15.32:0/3806662085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e0070c40 0x7f39e014f040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 >> 172.21.15.32:0/3806662085 conn(0x7f39e006c190 msgr2=0x7f39e00cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 shutdown_connections 2023-10-16T08:16:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.826+0000 7f39e7fc5640 1 -- 172.21.15.32:0/3806662085 wait complete. 2023-10-16T08:16:55.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff40d69e640 1 Processor -- start 2023-10-16T08:16:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff40d69e640 1 -- start start 2023-10-16T08:16:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff40d69e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff40815cb90 0x7ff4080cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff40d69e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080ce770 con 0x7ff4080d3010 2023-10-16T08:16:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff40d69e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080d0600 con 0x7ff4080d15d0 2023-10-16T08:16:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff40d69e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080c6450 con 0x7ff40815cb90 2023-10-16T08:16:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff3fffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4080d3010 0x7ff4080d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35872/0 (socket says 172.21.15.32:35872) 2023-10-16T08:16:55.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff3fffff640 1 -- 172.21.15.32:0/2079091977 learned_addr learned my addr 172.21.15.32:0/2079091977 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:55.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff406ffd640 1 --2- 172.21.15.32:0/2079091977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff40815cb90 0x7ff4080cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306008061 0 0) 0x7ff4080ce770 con 0x7ff4080d3010 2023-10-16T08:16:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f4003530 con 0x7ff4080d3010 2023-10-16T08:16:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978172148 0 0) 0x7ff4080d0600 con 0x7ff4080d15d0 2023-10-16T08:16:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4080ce770 con 0x7ff4080d15d0 2023-10-16T08:16:55.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff406ffd640 1 -- 172.21.15.32:0/2079091977 >> v1:172.21.15.186:6789/0 conn(0x7ff4080d15d0 legacy=0x7ff4080c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff406ffd640 1 -- 172.21.15.32:0/2079091977 >> v1:172.21.15.32:6789/0 conn(0x7ff4080d3010 legacy=0x7ff4080d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724412584 0 0) 0x7ff3f4003530 con 0x7ff4080d3010 2023-10-16T08:16:56.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.990+0000 7ff406ffd640 1 -- 172.21.15.32:0/2079091977 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080d08a0 con 0x7ff40815cb90 2023-10-16T08:16:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff406ffd640 1 --2- 172.21.15.32:0/2079091977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff40815cb90 0x7ff4080cfb10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80097d0 tx=0x7ff3f801c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ed2db392f26b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0001680 con 0x7ff40815cb90 2023-10-16T08:16:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f80a7050 con 0x7ff40815cb90 2023-10-16T08:16:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff4067fc640 1 -- 172.21.15.32:0/2079091977 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80ab620 con 0x7ff40815cb90 2023-10-16T08:16:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff40d69e640 1 -- 172.21.15.32:0/2079091977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff40815cb90 msgr2=0x7ff4080cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff40d69e640 1 --2- 172.21.15.32:0/2079091977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff40815cb90 0x7ff4080cfb10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80097d0 tx=0x7ff3f801c300 comp rx=0 tx=0).stop 2023-10-16T08:16:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff40d69e640 1 -- 172.21.15.32:0/2079091977 shutdown_connections 2023-10-16T08:16:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.994+0000 7ff40d69e640 1 --2- 172.21.15.32:0/2079091977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff40815cb90 0x7ff4080cfb10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- 172.21.15.32:0/2079091977 >> 172.21.15.32:0/2079091977 conn(0x7ff40809bc50 msgr2=0x7ff4080d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- 172.21.15.32:0/2079091977 shutdown_connections 2023-10-16T08:16:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- 172.21.15.32:0/2079091977 wait complete. 2023-10-16T08:16:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 Processor -- start 2023-10-16T08:16:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- start start 2023-10-16T08:16:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4080d15d0 0x7ff4081508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 0x7ff4080b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40815cb90 0x7ff4080b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40806c540 con 0x7ff4080d3010 2023-10-16T08:16:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408151050 con 0x7ff40815cb90 2023-10-16T08:16:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff40d69e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff408151330 con 0x7ff4080d15d0 2023-10-16T08:16:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 0x7ff4080b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 0x7ff4080b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45928/0 (socket says 172.21.15.32:45928) 2023-10-16T08:16:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 -- 172.21.15.32:0/1905628985 learned_addr learned my addr 172.21.15.32:0/1905628985 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff406ffd640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4080d15d0 0x7ff4081508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff4077fe640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40815cb90 0x7ff4080b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 -- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4080d15d0 msgr2=0x7ff4081508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4080d15d0 0x7ff4081508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 -- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40815cb90 msgr2=0x7ff4080b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40815cb90 0x7ff4080b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 -- 172.21.15.32:0/1905628985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80093b0 con 0x7ff4080d3010 2023-10-16T08:16:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff4077fe640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40815cb90 0x7ff4080b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:55.998+0000 7ff3fffff640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 0x7ff4080b76f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c750 tx=0x7ff3f000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0005ae0 con 0x7ff4080d3010 2023-10-16T08:16:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0005c40 con 0x7ff4080d3010 2023-10-16T08:16:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0007070 con 0x7ff4080d3010 2023-10-16T08:16:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff408151610 con 0x7ff4080d3010 2023-10-16T08:16:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4081536b0 con 0x7ff4080d3010 2023-10-16T08:16:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff408074010 con 0x7ff4080d3010 2023-10-16T08:16:56.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff3f00071d0 con 0x7ff4080d3010 2023-10-16T08:16:56.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff3ff7fe640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3e80664a0 0x7ff3e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff3f0083da0 con 0x7ff4080d3010 2023-10-16T08:16:56.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.002+0000 7ff406ffd640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3e80664a0 0x7ff3e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.006+0000 7ff406ffd640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3e80664a0 0x7ff3e8068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80ad000 tx=0x7ff3f80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.006+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff3f0050f20 con 0x7ff4080d3010 2023-10-16T08:16:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.166+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff408151ac0 con 0x7ff4080d3010 2023-10-16T08:16:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff3ff7fe640 1 -- 172.21.15.32:0/1905628985 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7ff3f0054540 con 0x7ff4080d3010 2023-10-16T08:16:56.174 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:56.174 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3e80664a0 msgr2=0x7ff3e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3e80664a0 0x7ff3e8068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80ad000 tx=0x7ff3f80a6000 comp rx=0 tx=0).stop 2023-10-16T08:16:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 msgr2=0x7ff4080b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 0x7ff4080b76f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c750 tx=0x7ff3f000cc10 comp rx=0 tx=0).stop 2023-10-16T08:16:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 shutdown_connections 2023-10-16T08:16:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff40815cb90 0x7ff4080b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4080d3010 0x7ff4080b76f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3e80664a0 0x7ff3e8068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 --2- 172.21.15.32:0/1905628985 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4080d15d0 0x7ff4081508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 >> 172.21.15.32:0/1905628985 conn(0x7ff40809bc50 msgr2=0x7ff4080cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 shutdown_connections 2023-10-16T08:16:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.170+0000 7ff40d69e640 1 -- 172.21.15.32:0/1905628985 wait complete. 2023-10-16T08:16:56.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:16:56.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:16:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe87e19d640 1 Processor -- start 2023-10-16T08:16:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe87e19d640 1 -- start start 2023-10-16T08:16:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe87e19d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 0x7fe8780d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe87e19d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8780d1670 con 0x7fe87815cba0 2023-10-16T08:16:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe87e19d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8780d1950 con 0x7fe8780d0210 2023-10-16T08:16:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe87e19d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780d1460 con 0x7fe8780b84e0 2023-10-16T08:16:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe87815cba0 0x7fe8780cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35874/0 (socket says 172.21.15.32:35874) 2023-10-16T08:16:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 -- 172.21.15.32:0/1900077745 learned_addr learned my addr 172.21.15.32:0/1900077745 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe877fff640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 0x7fe8780d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125390637 0 0) 0x7fe8780d1670 con 0x7fe87815cba0 2023-10-16T08:16:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe860003530 con 0x7fe87815cba0 2023-10-16T08:16:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3697709942 0 0) 0x7fe860003530 con 0x7fe87815cba0 2023-10-16T08:16:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 msgr2=0x7fe8780d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 0x7fe8780d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 >> v1:172.21.15.186:6789/0 conn(0x7fe8780d0210 legacy=0x7fe8780b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8780d1c30 con 0x7fe87815cba0 2023-10-16T08:16:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fe868001680 con 0x7fe87815cba0 2023-10-16T08:16:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 >> v1:172.21.15.32:6789/0 conn(0x7fe87815cba0 legacy=0x7fe8780cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe860003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 0x7fe8600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe860004d10 0x7fe860005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780d1460 con 0x7fe8600039b0 2023-10-16T08:16:56.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86000b990 con 0x7fe860004d10 2023-10-16T08:16:56.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86000bb50 con 0x7fe8780b84e0 2023-10-16T08:16:56.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe860003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe876ffd640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 0x7fe8600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe877fff640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe860004d10 0x7fe860005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 -- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 msgr2=0x7fe8600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8780b84e0 0x7fe8600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 -- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe860004d10 msgr2=0x7fe860005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe860004d10 0x7fe860005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.338+0000 7fe8777fe640 1 -- 172.21.15.32:0/1900077745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe860003530 con 0x7fe8600039b0 2023-10-16T08:16:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe8777fe640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe860003d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c190 tx=0x7fe86801cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc80131806cd9b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868005ae0 con 0x7fe8600039b0 2023-10-16T08:16:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868005c60 con 0x7fe8600039b0 2023-10-16T08:16:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe8767fc640 1 -- 172.21.15.32:0/1900077745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868007070 con 0x7fe8600039b0 2023-10-16T08:16:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe87e19d640 1 -- 172.21.15.32:0/1900077745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 msgr2=0x7fe860003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe87e19d640 1 --2- 172.21.15.32:0/1900077745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe860003d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c190 tx=0x7fe86801cb20 comp rx=0 tx=0).stop 2023-10-16T08:16:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe877fff640 1 -- 172.21.15.32:0/1900077745 reap_dead start 2023-10-16T08:16:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe87e19d640 1 -- 172.21.15.32:0/1900077745 shutdown_connections 2023-10-16T08:16:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe87e19d640 1 -- 172.21.15.32:0/1900077745 >> 172.21.15.32:0/1900077745 conn(0x7fe8780719c0 msgr2=0x7fe878071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe87e19d640 1 -- 172.21.15.32:0/1900077745 shutdown_connections 2023-10-16T08:16:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.342+0000 7fe87e19d640 1 -- 172.21.15.32:0/1900077745 wait complete. 2023-10-16T08:16:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 Processor -- start 2023-10-16T08:16:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 -- start start 2023-10-16T08:16:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe8780cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87815faa0 0x7fe87814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87815ff00 0x7fe8780c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780c79a0 con 0x7fe8600039b0 2023-10-16T08:16:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780c7c80 con 0x7fe87815ff00 2023-10-16T08:16:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780c7f60 con 0x7fe87815faa0 2023-10-16T08:16:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe8780cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe8780cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45948/0 (socket says 172.21.15.32:45948) 2023-10-16T08:16:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 -- 172.21.15.32:0/322732362 learned_addr learned my addr 172.21.15.32:0/322732362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe876ffd640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87815faa0 0x7fe87814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe877fff640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87815ff00 0x7fe8780c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 -- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87815faa0 msgr2=0x7fe87814f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87815faa0 0x7fe87814f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 -- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87815ff00 msgr2=0x7fe8780c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87815ff00 0x7fe8780c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 -- 172.21.15.32:0/322732362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8780c8240 con 0x7fe8600039b0 2023-10-16T08:16:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe877fff640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87815ff00 0x7fe8780c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe8777fe640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe8780cc5e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe86801c950 tx=0x7fe860005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680b4070 con 0x7fe8600039b0 2023-10-16T08:16:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868001440 con 0x7fe8600039b0 2023-10-16T08:16:56.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680070a0 con 0x7fe8600039b0 2023-10-16T08:16:56.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8780c8520 con 0x7fe8600039b0 2023-10-16T08:16:56.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.346+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8780c7650 con 0x7fe8600039b0 2023-10-16T08:16:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.350+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8780c8710 con 0x7fe8600039b0 2023-10-16T08:16:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.350+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe868007200 con 0x7fe8600039b0 2023-10-16T08:16:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.350+0000 7fe85ffff640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8640664a0 0x7fe864068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.350+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe86811da50 con 0x7fe8600039b0 2023-10-16T08:16:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.350+0000 7fe876ffd640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8640664a0 0x7fe864068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.350+0000 7fe876ffd640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8640664a0 0x7fe864068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8780731c0 tx=0x7fe86c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.354+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe8680eabe0 con 0x7fe8600039b0 2023-10-16T08:16:56.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.510+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8780c8710 con 0x7fe8600039b0 2023-10-16T08:16:56.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.510+0000 7fe85ffff640 1 -- 172.21.15.32:0/322732362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8680ee200 con 0x7fe8600039b0 2023-10-16T08:16:56.516 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:16:56.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.510+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8640664a0 msgr2=0x7fe864068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.510+0000 7fe87e19d640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8640664a0 0x7fe864068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8780731c0 tx=0x7fe86c009040 comp rx=0 tx=0).stop 2023-10-16T08:16:56.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.510+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 msgr2=0x7fe8780cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.510+0000 7fe87e19d640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe8780cc5e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe86801c950 tx=0x7fe860005800 comp rx=0 tx=0).stop 2023-10-16T08:16:56.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 shutdown_connections 2023-10-16T08:16:56.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe87815ff00 0x7fe8780c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8640664a0 0x7fe864068960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe87815faa0 0x7fe87814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 --2- 172.21.15.32:0/322732362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8600039b0 0x7fe8780cc5e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 >> 172.21.15.32:0/322732362 conn(0x7fe8780719c0 msgr2=0x7fe878157880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:56.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 shutdown_connections 2023-10-16T08:16:56.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.514+0000 7fe87e19d640 1 -- 172.21.15.32:0/322732362 wait complete. 2023-10-16T08:16:56.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:16:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 Processor -- start 2023-10-16T08:16:56.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 -- start start 2023-10-16T08:16:56.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e815cc60 0x7f10e80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10e8161260 0x7f10e80ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f10e80cebd0 0x7f10e80b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e80cc330 con 0x7f10e8161260 2023-10-16T08:16:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e80cefe0 con 0x7f10e80cebd0 2023-10-16T08:16:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10eed2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80cb720 con 0x7f10e815cc60 2023-10-16T08:16:56.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10e8161260 0x7f10e80ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10e8161260 0x7f10e80ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45950/0 (socket says 172.21.15.32:45950) 2023-10-16T08:16:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 -- 172.21.15.32:0/94622843 learned_addr learned my addr 172.21.15.32:0/94622843 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10ecac6640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e815cc60 0x7f10e80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10ed2c7640 1 --2- 172.21.15.32:0/94622843 >> v2:172.21.15.186:3300/0 conn(0x7f10e80cebd0 0x7f10e80b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 -- 172.21.15.32:0/94622843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e815cc60 msgr2=0x7f10e80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e815cc60 0x7f10e80cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 -- 172.21.15.32:0/94622843 >> v2:172.21.15.186:3300/0 conn(0x7f10e80cebd0 msgr2=0x7f10e80b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 --2- 172.21.15.32:0/94622843 >> v2:172.21.15.186:3300/0 conn(0x7f10e80cebd0 0x7f10e80b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10e7fff640 1 -- 172.21.15.32:0/94622843 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80cba00 con 0x7f10e8161260 2023-10-16T08:16:56.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.678+0000 7f10ecac6640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e815cc60 0x7f10e80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10e7fff640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e8161260 0x7f10e80ce690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f10e80cdf60 tx=0x7f10d401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1035adc844535 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10e77fe640 1 -- 172.21.15.32:0/94622843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40ab600 con 0x7f10e8161260 2023-10-16T08:16:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10e77fe640 1 -- 172.21.15.32:0/94622843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40a7050 con 0x7f10e8161260 2023-10-16T08:16:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10e77fe640 1 -- 172.21.15.32:0/94622843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40ab980 con 0x7f10e8161260 2023-10-16T08:16:56.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 -- 172.21.15.32:0/94622843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e8161260 msgr2=0x7f10e80ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e8161260 0x7f10e80ce690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f10e80cdf60 tx=0x7f10d401c210 comp rx=0 tx=0).stop 2023-10-16T08:16:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 -- 172.21.15.32:0/94622843 shutdown_connections 2023-10-16T08:16:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 --2- 172.21.15.32:0/94622843 >> v2:172.21.15.186:3300/0 conn(0x7f10e80cebd0 0x7f10e80b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e8161260 0x7f10e80ce690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 --2- 172.21.15.32:0/94622843 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e815cc60 0x7f10e80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 -- 172.21.15.32:0/94622843 >> 172.21.15.32:0/94622843 conn(0x7f10e806e570 msgr2=0x7f10e806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 -- 172.21.15.32:0/94622843 shutdown_connections 2023-10-16T08:16:56.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.682+0000 7f10eed2a640 1 -- 172.21.15.32:0/94622843 wait complete. 2023-10-16T08:16:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 Processor -- start 2023-10-16T08:16:56.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 -- start start 2023-10-16T08:16:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e80cebd0 0x7f10e814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 0x7f10e8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e8161260 0x7f10e8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e815b5a0 con 0x7f10e815cc60 2023-10-16T08:16:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e815b9c0 con 0x7f10e80cebd0 2023-10-16T08:16:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8150530 con 0x7f10e8161260 2023-10-16T08:16:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 0x7f10e8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 0x7f10e8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45952/0 (socket says 172.21.15.32:45952) 2023-10-16T08:16:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 -- 172.21.15.32:0/463795971 learned_addr learned my addr 172.21.15.32:0/463795971 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10ecac6640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e80cebd0 0x7f10e814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10ed2c7640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e8161260 0x7f10e8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 -- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e8161260 msgr2=0x7f10e8157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e8161260 0x7f10e8157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 -- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e80cebd0 msgr2=0x7f10e814fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e80cebd0 0x7f10e814fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 -- 172.21.15.32:0/463795971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40094d0 con 0x7f10e815cc60 2023-10-16T08:16:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e7fff640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 0x7f10e8156fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10d40ae040 tx=0x7f10d4018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40b4620 con 0x7f10e815cc60 2023-10-16T08:16:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40b4780 con 0x7f10e815cc60 2023-10-16T08:16:56.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e815a920 con 0x7f10e815cc60 2023-10-16T08:16:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40b4b30 con 0x7f10e815cc60 2023-10-16T08:16:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.686+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e815aee0 con 0x7f10e815cc60 2023-10-16T08:16:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.690+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e815b6e0 con 0x7f10e815cc60 2023-10-16T08:16:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.690+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10d40b2070 con 0x7f10e815cc60 2023-10-16T08:16:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.690+0000 7f10e57fa640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10cc0664a0 0x7f10cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.690+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f10d411d0f0 con 0x7f10e815cc60 2023-10-16T08:16:56.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.690+0000 7f10ecac6640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10cc0664a0 0x7f10cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:56.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.690+0000 7f10ecac6640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10cc0664a0 0x7f10cc068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10e81596d0 tx=0x7f10dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:56.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.694+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10e815b6e0 con 0x7f10e815cc60 2023-10-16T08:16:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10e815b6e0 con 0x7f10e815cc60 2023-10-16T08:16:56.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10e57fa640 1 -- 172.21.15.32:0/463795971 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10e815b6e0 con 0x7f10e815cc60 2023-10-16T08:16:56.863 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:56.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10cc0664a0 msgr2=0x7f10cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10cc0664a0 0x7f10cc068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10e81596d0 tx=0x7f10dc008040 comp rx=0 tx=0).stop 2023-10-16T08:16:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 msgr2=0x7f10e8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 0x7f10e8156fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10d40ae040 tx=0x7f10d4018fb0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 shutdown_connections 2023-10-16T08:16:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10cc0664a0 0x7f10cc068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10e8161260 0x7f10e8157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e815cc60 0x7f10e8156fd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 --2- 172.21.15.32:0/463795971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e80cebd0 0x7f10e814fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 >> 172.21.15.32:0/463795971 conn(0x7f10e806e570 msgr2=0x7f10e80b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 shutdown_connections 2023-10-16T08:16:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:56.858+0000 7f10eed2a640 1 -- 172.21.15.32:0/463795971 wait complete. 2023-10-16T08:16:56.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:16:57.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.022+0000 7ff0ac46b640 1 Processor -- start 2023-10-16T08:16:57.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 -- start start 2023-10-16T08:16:57.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0a415cba0 0x7ff0a40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff0a40b84e0 0x7ff0a40cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a40d0040 0x7ff0a40d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a40d1460 con 0x7ff0a415cba0 2023-10-16T08:16:57.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a40d04b0 con 0x7ff0a40b84e0 2023-10-16T08:16:57.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0ac46b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40d1670 con 0x7ff0a40d0040 2023-10-16T08:16:57.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0a415cba0 0x7ff0a40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0a415cba0 0x7ff0a40d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45956/0 (socket says 172.21.15.32:45956) 2023-10-16T08:16:57.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 -- 172.21.15.32:0/3889794300 learned_addr learned my addr 172.21.15.32:0/3889794300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:57.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 -- 172.21.15.32:0/3889794300 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a40d0040 msgr2=0x7ff0a40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:57.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 --2- 172.21.15.32:0/3889794300 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a40d0040 0x7ff0a40d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 -- 172.21.15.32:0/3889794300 >> v2:172.21.15.186:3300/0 conn(0x7ff0a40b84e0 msgr2=0x7ff0a40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:57.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 --2- 172.21.15.32:0/3889794300 >> v2:172.21.15.186:3300/0 conn(0x7ff0a40b84e0 0x7ff0a40cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.026+0000 7ff0aa207640 1 -- 172.21.15.32:0/3889794300 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a40d1950 con 0x7ff0a415cba0 2023-10-16T08:16:57.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0aa207640 1 --2- 172.21.15.32:0/3889794300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a415cba0 0x7ff0a40d0f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40cf2f0 tx=0x7ff0a001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801c7e119cc1b483 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0a9205640 1 -- 172.21.15.32:0/3889794300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00ab600 con 0x7ff0a415cba0 2023-10-16T08:16:57.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0a9205640 1 -- 172.21.15.32:0/3889794300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a00a7050 con 0x7ff0a415cba0 2023-10-16T08:16:57.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0a9205640 1 -- 172.21.15.32:0/3889794300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00ab980 con 0x7ff0a415cba0 2023-10-16T08:16:57.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 -- 172.21.15.32:0/3889794300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a415cba0 msgr2=0x7ff0a40d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/3889794300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a415cba0 0x7ff0a40d0f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40cf2f0 tx=0x7ff0a001c240 comp rx=0 tx=0).stop 2023-10-16T08:16:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 -- 172.21.15.32:0/3889794300 shutdown_connections 2023-10-16T08:16:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/3889794300 >> v2:172.21.15.186:3300/0 conn(0x7ff0a40b84e0 0x7ff0a40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/3889794300 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a40d0040 0x7ff0a40d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/3889794300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a415cba0 0x7ff0a40d0f20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 -- 172.21.15.32:0/3889794300 >> 172.21.15.32:0/3889794300 conn(0x7ff0a40719c0 msgr2=0x7ff0a4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:57.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 -- 172.21.15.32:0/3889794300 shutdown_connections 2023-10-16T08:16:57.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 -- 172.21.15.32:0/3889794300 wait complete. 2023-10-16T08:16:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 Processor -- start 2023-10-16T08:16:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.030+0000 7ff0ac46b640 1 -- start start 2023-10-16T08:16:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 0x7ff0a415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a40d0040 0x7ff0a4151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a415cba0 0x7ff0a4155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a41601d0 con 0x7ff0a40b84e0 2023-10-16T08:16:57.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4152120 con 0x7ff0a40d0040 2023-10-16T08:16:57.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a4155980 con 0x7ff0a415cba0 2023-10-16T08:16:57.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 0x7ff0a415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 0x7ff0a415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45966/0 (socket says 172.21.15.32:45966) 2023-10-16T08:16:57.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 -- 172.21.15.32:0/74445700 learned_addr learned my addr 172.21.15.32:0/74445700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:57.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aaa08640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a415cba0 0x7ff0a4155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0a9a06640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a40d0040 0x7ff0a4151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 -- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a415cba0 msgr2=0x7ff0a4155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a415cba0 0x7ff0a4155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 -- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a40d0040 msgr2=0x7ff0a4151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a40d0040 0x7ff0a4151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 -- 172.21.15.32:0/74445700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00094d0 con 0x7ff0a40b84e0 2023-10-16T08:16:57.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0aa207640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 0x7ff0a415fc90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff0a001cae0 tx=0x7ff0a001aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00b6620 con 0x7ff0a40b84e0 2023-10-16T08:16:57.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a00a4e90 con 0x7ff0a40b84e0 2023-10-16T08:16:57.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00b69e0 con 0x7ff0a40b84e0 2023-10-16T08:16:57.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a4156350 con 0x7ff0a40b84e0 2023-10-16T08:16:57.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a4155330 con 0x7ff0a40b84e0 2023-10-16T08:16:57.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a404e170 con 0x7ff0a40b84e0 2023-10-16T08:16:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.034+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff0a00b2040 con 0x7ff0a40b84e0 2023-10-16T08:16:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.038+0000 7ff09affd640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0880664a0 0x7ff088068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.038+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0a011e310 con 0x7ff0a40b84e0 2023-10-16T08:16:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.038+0000 7ff0a9a06640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0880664a0 0x7ff088068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.038+0000 7ff0a9a06640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0880664a0 0x7ff088068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff094006fd0 tx=0x7ff094008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.042+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0a00eb520 con 0x7ff0a40b84e0 2023-10-16T08:16:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.198+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0a40c94b0 con 0x7ff0a40b84e0 2023-10-16T08:16:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.198+0000 7ff09affd640 1 -- 172.21.15.32:0/74445700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0a00eeb40 con 0x7ff0a40b84e0 2023-10-16T08:16:57.205 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:16:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0880664a0 msgr2=0x7ff088068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0880664a0 0x7ff088068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff094006fd0 tx=0x7ff094008040 comp rx=0 tx=0).stop 2023-10-16T08:16:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 msgr2=0x7ff0a415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 0x7ff0a415fc90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff0a001cae0 tx=0x7ff0a001aa80 comp rx=0 tx=0).stop 2023-10-16T08:16:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 shutdown_connections 2023-10-16T08:16:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0880664a0 0x7ff088068960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0a415cba0 0x7ff0a4155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0a40d0040 0x7ff0a4151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 --2- 172.21.15.32:0/74445700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0a40b84e0 0x7ff0a415fc90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 >> 172.21.15.32:0/74445700 conn(0x7ff0a40719c0 msgr2=0x7ff0a40b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 shutdown_connections 2023-10-16T08:16:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.202+0000 7ff0ac46b640 1 -- 172.21.15.32:0/74445700 wait complete. 2023-10-16T08:16:57.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:16:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 Processor -- start 2023-10-16T08:16:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 -- start start 2023-10-16T08:16:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd18c0b84e0 0x7fd18c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0d1670 con 0x7fd18c0d0040 2023-10-16T08:16:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0d1460 con 0x7fd18c0b84e0 2023-10-16T08:16:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd1943c5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0d04b0 con 0x7fd18c15cba0 2023-10-16T08:16:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd18c0b84e0 0x7fd18c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd18c0b84e0 0x7fd18c0cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45970/0 (socket says 172.21.15.32:45970) 2023-10-16T08:16:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd192962640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd18c0d0040 0x7fd18c0d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35888/0 (socket says 172.21.15.32:35888) 2023-10-16T08:16:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 -- 172.21.15.32:0/668310421 learned_addr learned my addr 172.21.15.32:0/668310421 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd192161640 1 --2- 172.21.15.32:0/668310421 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 -- 172.21.15.32:0/668310421 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 msgr2=0x7fd18c0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 --2- 172.21.15.32:0/668310421 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 -- 172.21.15.32:0/668310421 >> v1:172.21.15.32:6789/0 conn(0x7fd18c0d0040 legacy=0x7fd18c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.366+0000 7fd191960640 1 -- 172.21.15.32:0/668310421 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0d1950 con 0x7fd18c0b84e0 2023-10-16T08:16:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd191960640 1 --2- 172.21.15.32:0/668310421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0b84e0 0x7fd18c0cfb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87547253d4ec1cfe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd19115f640 1 -- 172.21.15.32:0/668310421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0ab600 con 0x7fd18c0b84e0 2023-10-16T08:16:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd19115f640 1 -- 172.21.15.32:0/668310421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0a7050 con 0x7fd18c0b84e0 2023-10-16T08:16:57.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd19115f640 1 -- 172.21.15.32:0/668310421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0ab980 con 0x7fd18c0b84e0 2023-10-16T08:16:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 -- 172.21.15.32:0/668310421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0b84e0 msgr2=0x7fd18c0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 --2- 172.21.15.32:0/668310421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0b84e0 0x7fd18c0cfb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c01c3d0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 -- 172.21.15.32:0/668310421 shutdown_connections 2023-10-16T08:16:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 --2- 172.21.15.32:0/668310421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0b84e0 0x7fd18c0cfb00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 --2- 172.21.15.32:0/668310421 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 -- 172.21.15.32:0/668310421 >> 172.21.15.32:0/668310421 conn(0x7fd18c0719c0 msgr2=0x7fd18c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 -- 172.21.15.32:0/668310421 shutdown_connections 2023-10-16T08:16:57.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.370+0000 7fd1943c5640 1 -- 172.21.15.32:0/668310421 wait complete. 2023-10-16T08:16:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 Processor -- start 2023-10-16T08:16:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 -- start start 2023-10-16T08:16:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b84e0 0x7fd18c1468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 0x7fd18c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c143f00 con 0x7fd18c0d0040 2023-10-16T08:16:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c146de0 con 0x7fd18c0b84e0 2023-10-16T08:16:57.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c1f5c60 con 0x7fd18c15cba0 2023-10-16T08:16:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 0x7fd18c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 0x7fd18c15e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45972/0 (socket says 172.21.15.32:45972) 2023-10-16T08:16:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 -- 172.21.15.32:0/398477202 learned_addr learned my addr 172.21.15.32:0/398477202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd192161640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b84e0 0x7fd18c1468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd192962640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 -- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 msgr2=0x7fd18c0c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 -- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b84e0 msgr2=0x7fd18c1468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b84e0 0x7fd18c1468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 -- 172.21.15.32:0/398477202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0094d0 con 0x7fd18c0d0040 2023-10-16T08:16:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd192962640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd191960640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 0x7fd18c15e290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01cc70 tx=0x7fd17c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0b4850 con 0x7fd18c0d0040 2023-10-16T08:16:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0b49b0 con 0x7fd18c0d0040 2023-10-16T08:16:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0b4d60 con 0x7fd18c0d0040 2023-10-16T08:16:57.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c1f5f40 con 0x7fd18c0d0040 2023-10-16T08:16:57.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.374+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c1f6500 con 0x7fd18c0d0040 2023-10-16T08:16:57.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.378+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c144040 con 0x7fd18c0d0040 2023-10-16T08:16:57.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.378+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd17c0b2070 con 0x7fd18c0d0040 2023-10-16T08:16:57.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.378+0000 7fd182ffd640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1700664a0 0x7fd170068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.378+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd17c11e4d0 con 0x7fd18c0d0040 2023-10-16T08:16:57.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.378+0000 7fd192161640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1700664a0 0x7fd170068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.378+0000 7fd192161640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1700664a0 0x7fd170068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd18800b9d0 tx=0x7fd188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.382+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd18c144040 con 0x7fd18c0d0040 2023-10-16T08:16:57.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd18c144040 con 0x7fd18c0d0040 2023-10-16T08:16:57.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd182ffd640 1 -- 172.21.15.32:0/398477202 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd18c144040 con 0x7fd18c0d0040 2023-10-16T08:16:57.547 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:16:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1700664a0 msgr2=0x7fd170068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1700664a0 0x7fd170068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd18800b9d0 tx=0x7fd188008040 comp rx=0 tx=0).stop 2023-10-16T08:16:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 msgr2=0x7fd18c15e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 0x7fd18c15e290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01cc70 tx=0x7fd17c019170 comp rx=0 tx=0).stop 2023-10-16T08:16:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 shutdown_connections 2023-10-16T08:16:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1700664a0 0x7fd170068960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd18c15cba0 0x7fd18c0c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd18c0d0040 0x7fd18c15e290 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 --2- 172.21.15.32:0/398477202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd18c0b84e0 0x7fd18c1468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 >> 172.21.15.32:0/398477202 conn(0x7fd18c0719c0 msgr2=0x7fd18c157490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 shutdown_connections 2023-10-16T08:16:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.542+0000 7fd1943c5640 1 -- 172.21.15.32:0/398477202 wait complete. 2023-10-16T08:16:57.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 Processor -- start 2023-10-16T08:16:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 -- start start 2023-10-16T08:16:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415c980 0x7fc0b406af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc0b40d1490 0x7fc0b40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b40d3510 con 0x7fc0b415cd60 2023-10-16T08:16:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b40cd2b0 con 0x7fc0b40d1490 2023-10-16T08:16:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b9162640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40d2f50 con 0x7fc0b415c980 2023-10-16T08:16:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b37fe640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fc0b415cd60 0x7fc0b40d0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:48702/0 (socket says 172.21.15.32:48702) 2023-10-16T08:16:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b37fe640 1 -- 172.21.15.32:0/962919647 learned_addr learned my addr 172.21.15.32:0/962919647 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b3fff640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415c980 0x7fc0b406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b8961640 1 --2- 172.21.15.32:0/962919647 >> v2:172.21.15.186:3300/0 conn(0x7fc0b40d1490 0x7fc0b40b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b2ffd640 1 -- 172.21.15.32:0/962919647 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181727907 0 0) 0x7fc0a8001120 con 0x7fc0b415cd60 2023-10-16T08:16:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b2ffd640 1 -- 172.21.15.32:0/962919647 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09c003530 con 0x7fc0b415cd60 2023-10-16T08:16:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b8961640 1 -- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415c980 msgr2=0x7fc0b406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b8961640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415c980 0x7fc0b406af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b8961640 1 -- 172.21.15.32:0/962919647 >> v1:172.21.15.186:6789/0 conn(0x7fc0b415cd60 legacy=0x7fc0b40d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b8961640 1 -- 172.21.15.32:0/962919647 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b40d3230 con 0x7fc0b40d1490 2023-10-16T08:16:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.706+0000 7fc0b3fff640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415c980 0x7fc0b406af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:16:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b8961640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b40b1b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009dc0 tx=0x7fc0ac01c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f288129b3cdebe5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b2ffd640 1 -- 172.21.15.32:0/962919647 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a5900 con 0x7fc0b40d1490 2023-10-16T08:16:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b2ffd640 1 -- 172.21.15.32:0/962919647 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0a5a60 con 0x7fc0b40d1490 2023-10-16T08:16:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b2ffd640 1 -- 172.21.15.32:0/962919647 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a5df0 con 0x7fc0b40d1490 2023-10-16T08:16:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 -- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 msgr2=0x7fc0b40b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b40b1b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009dc0 tx=0x7fc0ac01c930 comp rx=0 tx=0).stop 2023-10-16T08:16:57.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 -- 172.21.15.32:0/962919647 shutdown_connections 2023-10-16T08:16:57.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b40b1b90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 --2- 172.21.15.32:0/962919647 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415c980 0x7fc0b406af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 -- 172.21.15.32:0/962919647 >> 172.21.15.32:0/962919647 conn(0x7fc0b409ba80 msgr2=0x7fc0b40d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:57.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 -- 172.21.15.32:0/962919647 shutdown_connections 2023-10-16T08:16:57.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.710+0000 7fc0b9162640 1 -- 172.21.15.32:0/962919647 wait complete. 2023-10-16T08:16:57.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 Processor -- start 2023-10-16T08:16:57.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- start start 2023-10-16T08:16:57.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b414fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 0x7fc0b4166ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415cd60 0x7fc0b40af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4166ff0 con 0x7fc0b415c980 2023-10-16T08:16:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4165a10 con 0x7fc0b40d1490 2023-10-16T08:16:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4165cf0 con 0x7fc0b415cd60 2023-10-16T08:16:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 0x7fc0b4166ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 0x7fc0b4166ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45976/0 (socket says 172.21.15.32:45976) 2023-10-16T08:16:57.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 -- 172.21.15.32:0/2928646597 learned_addr learned my addr 172.21.15.32:0/2928646597 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:57.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b3fff640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b414fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b8961640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415cd60 0x7fc0b40af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 -- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415cd60 msgr2=0x7fc0b40af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415cd60 0x7fc0b40af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 -- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 msgr2=0x7fc0b414fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b414fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 -- 172.21.15.32:0/2928646597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40093b0 con 0x7fc0b415c980 2023-10-16T08:16:57.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b37fe640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 0x7fc0b4166ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c420 tx=0x7fc0a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8005cf0 con 0x7fc0b415c980 2023-10-16T08:16:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8005e50 con 0x7fc0b415c980 2023-10-16T08:16:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80072f0 con 0x7fc0b415c980 2023-10-16T08:16:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac009a70 con 0x7fc0b415c980 2023-10-16T08:16:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b41662b0 con 0x7fc0b415c980 2023-10-16T08:16:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.714+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b404e9c0 con 0x7fc0b415c980 2023-10-16T08:16:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.718+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc0a8007450 con 0x7fc0b415c980 2023-10-16T08:16:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.718+0000 7fc0b0ff9640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0a00664a0 0x7fc0a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.718+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc0a8083520 con 0x7fc0b415c980 2023-10-16T08:16:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.718+0000 7fc0b3fff640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0a00664a0 0x7fc0a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.718+0000 7fc0b3fff640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0a00664a0 0x7fc0a0068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009cd0 tx=0x7fc0a4006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:57.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.722+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc0a8050750 con 0x7fc0b415c980 2023-10-16T08:16:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0b40cff00 con 0x7fc0b415c980 2023-10-16T08:16:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b0ff9640 1 -- 172.21.15.32:0/2928646597 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7fc0a8053d70 con 0x7fc0b415c980 2023-10-16T08:16:57.920 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:57.920 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:16:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:16:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0a00664a0 msgr2=0x7fc0a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0a00664a0 0x7fc0a0068960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009cd0 tx=0x7fc0a4006f90 comp rx=0 tx=0).stop 2023-10-16T08:16:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 msgr2=0x7fc0b4166ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 0x7fc0b4166ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c420 tx=0x7fc0a800c8e0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 shutdown_connections 2023-10-16T08:16:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0a00664a0 0x7fc0a0068960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0b415cd60 0x7fc0b40af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b415c980 0x7fc0b4166ab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 --2- 172.21.15.32:0/2928646597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0b40d1490 0x7fc0b414fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 >> 172.21.15.32:0/2928646597 conn(0x7fc0b409ba80 msgr2=0x7fc0b40c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 shutdown_connections 2023-10-16T08:16:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:57.914+0000 7fc0b9162640 1 -- 172.21.15.32:0/2928646597 wait complete. 2023-10-16T08:16:57.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:16:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.078+0000 7fbfa0409640 1 Processor -- start 2023-10-16T08:16:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.078+0000 7fbfa0409640 1 -- start start 2023-10-16T08:16:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbfa0409640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf980ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbfa0409640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf980adc10 0x7fbf980cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbfa0409640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf980723d0 con 0x7fbf98159ba0 2023-10-16T08:16:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbfa0409640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf980717c0 con 0x7fbf980adc10 2023-10-16T08:16:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbfa0409640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf98071aa0 con 0x7fbf981597c0 2023-10-16T08:16:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf980adc10 0x7fbf980cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf980adc10 0x7fbf980cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45978/0 (socket says 172.21.15.32:45978) 2023-10-16T08:16:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1657104809 learned_addr learned my addr 172.21.15.32:0/1657104809 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e1a5640 1 --2- 172.21.15.32:0/1657104809 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf980ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1657104809 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 msgr2=0x7fbf980ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 --2- 172.21.15.32:0/1657104809 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf980ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1657104809 >> v1:172.21.15.186:6789/0 conn(0x7fbf98159ba0 legacy=0x7fbf980cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1657104809 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98071d80 con 0x7fbf980adc10 2023-10-16T08:16:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9e9a6640 1 --2- 172.21.15.32:0/1657104809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf980adc10 0x7fbf980cb520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009e50 tx=0x7fbf9001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6afee75562e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.082+0000 7fbf9d1a3640 1 -- 172.21.15.32:0/1657104809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900a5880 con 0x7fbf980adc10 2023-10-16T08:16:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9d1a3640 1 -- 172.21.15.32:0/1657104809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf900a59e0 con 0x7fbf980adc10 2023-10-16T08:16:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9d1a3640 1 -- 172.21.15.32:0/1657104809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900a5d70 con 0x7fbf980adc10 2023-10-16T08:16:58.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- 172.21.15.32:0/1657104809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf980adc10 msgr2=0x7fbf980cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1657104809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf980adc10 0x7fbf980cb520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009e50 tx=0x7fbf9001ca10 comp rx=0 tx=0).stop 2023-10-16T08:16:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- 172.21.15.32:0/1657104809 shutdown_connections 2023-10-16T08:16:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1657104809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf980adc10 0x7fbf980cb520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1657104809 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf980ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- 172.21.15.32:0/1657104809 >> 172.21.15.32:0/1657104809 conn(0x7fbf9806c190 msgr2=0x7fbf9806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:58.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- 172.21.15.32:0/1657104809 shutdown_connections 2023-10-16T08:16:58.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- 172.21.15.32:0/1657104809 wait complete. 2023-10-16T08:16:58.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 Processor -- start 2023-10-16T08:16:58.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- start start 2023-10-16T08:16:58.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf980adc10 0x7fbf98156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf98149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 0x7fbf9815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981f8f20 con 0x7fbf98159ba0 2023-10-16T08:16:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9815cd10 con 0x7fbf980adc10 2023-10-16T08:16:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbfa0409640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981f91e0 con 0x7fbf981597c0 2023-10-16T08:16:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9e9a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 0x7fbf9815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9e9a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 0x7fbf9815c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45994/0 (socket says 172.21.15.32:45994) 2023-10-16T08:16:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1153004356 learned_addr learned my addr 172.21.15.32:0/1153004356 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9e1a5640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf980adc10 0x7fbf98156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.086+0000 7fbf9d9a4640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf98149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 msgr2=0x7fbf98149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf9e9a6640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf98149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf980adc10 msgr2=0x7fbf98156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf9e9a6640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf980adc10 0x7fbf98156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf9e9a6640 1 -- 172.21.15.32:0/1153004356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90009b30 con 0x7fbf98159ba0 2023-10-16T08:16:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf9e9a6640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 0x7fbf9815c680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf900af040 tx=0x7fbf900197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900b8670 con 0x7fbf98159ba0 2023-10-16T08:16:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf900b87d0 con 0x7fbf98159ba0 2023-10-16T08:16:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf981f94c0 con 0x7fbf98159ba0 2023-10-16T08:16:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900b8b60 con 0x7fbf98159ba0 2023-10-16T08:16:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf981561b0 con 0x7fbf98159ba0 2023-10-16T08:16:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf981f99e0 con 0x7fbf98159ba0 2023-10-16T08:16:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbf900b8cc0 con 0x7fbf98159ba0 2023-10-16T08:16:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.090+0000 7fbf8effd640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbf7c0664d0 0x7fbf7c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.094+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf9011f030 con 0x7fbf98159ba0 2023-10-16T08:16:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.094+0000 7fbf9e1a5640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbf7c0664d0 0x7fbf7c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.094+0000 7fbf9e1a5640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbf7c0664d0 0x7fbf7c068990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009b60 tx=0x7fbf94009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.098+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbf981f99e0 con 0x7fbf98159ba0 2023-10-16T08:16:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.254+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf981f99e0 con 0x7fbf98159ba0 2023-10-16T08:16:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.254+0000 7fbf8effd640 1 -- 172.21.15.32:0/1153004356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fbf9011c7c0 con 0x7fbf98159ba0 2023-10-16T08:16:58.260 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:58.261 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:16:27.395838+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:16:58.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbf7c0664d0 msgr2=0x7fbf7c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbf7c0664d0 0x7fbf7c068990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009b60 tx=0x7fbf94009190 comp rx=0 tx=0).stop 2023-10-16T08:16:58.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 msgr2=0x7fbf9815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 0x7fbf9815c680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf900af040 tx=0x7fbf900197b0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 shutdown_connections 2023-10-16T08:16:58.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf98159ba0 0x7fbf9815c680 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbf7c0664d0 0x7fbf7c068990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf981597c0 0x7fbf98149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 --2- 172.21.15.32:0/1153004356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf980adc10 0x7fbf98156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 >> 172.21.15.32:0/1153004356 conn(0x7fbf9806c190 msgr2=0x7fbf980ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:58.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 shutdown_connections 2023-10-16T08:16:58.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.258+0000 7fbfa0409640 1 -- 172.21.15.32:0/1153004356 wait complete. 2023-10-16T08:16:58.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:16:58.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.418+0000 7f5c0e086640 1 Processor -- start 2023-10-16T08:16:58.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.418+0000 7f5c0e086640 1 -- start start 2023-10-16T08:16:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c0e086640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0815ca70 0x7f5c080d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c0e086640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c080d3550 0x7f5c080cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c0e086640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c080c65f0 con 0x7f5c080cf5d0 2023-10-16T08:16:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c0e086640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c080ce820 con 0x7f5c080d3550 2023-10-16T08:16:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c0e086640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080cfa40 con 0x7f5c0815ca70 2023-10-16T08:16:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c080d3550 0x7f5c080cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c080d3550 0x7f5c080cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45998/0 (socket says 172.21.15.32:45998) 2023-10-16T08:16:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 -- 172.21.15.32:0/2494657487 learned_addr learned my addr 172.21.15.32:0/2494657487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c077fe640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0815ca70 0x7f5c080d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 -- 172.21.15.32:0/2494657487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0815ca70 msgr2=0x7f5c080d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0815ca70 0x7f5c080d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 -- 172.21.15.32:0/2494657487 >> v1:172.21.15.186:6789/0 conn(0x7f5c080cf5d0 legacy=0x7f5c080d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 -- 172.21.15.32:0/2494657487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080cdf80 con 0x7f5c080d3550 2023-10-16T08:16:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c077fe640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0815ca70 0x7f5c080d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:58.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c06ffd640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c080d3550 0x7f5c080cf090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0098a0 tx=0x7f5bfc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d794155f17e4185 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.422+0000 7f5c067fc640 1 -- 172.21.15.32:0/2494657487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0aa650 con 0x7f5c080d3550 2023-10-16T08:16:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c067fc640 1 -- 172.21.15.32:0/2494657487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc0aa7b0 con 0x7f5c080d3550 2023-10-16T08:16:58.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c067fc640 1 -- 172.21.15.32:0/2494657487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0aab40 con 0x7f5c080d3550 2023-10-16T08:16:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- 172.21.15.32:0/2494657487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c080d3550 msgr2=0x7f5c080cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c080d3550 0x7f5c080cf090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0098a0 tx=0x7f5bfc01c4f0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- 172.21.15.32:0/2494657487 shutdown_connections 2023-10-16T08:16:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c080d3550 0x7f5c080cf090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 --2- 172.21.15.32:0/2494657487 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c0815ca70 0x7f5c080d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- 172.21.15.32:0/2494657487 >> 172.21.15.32:0/2494657487 conn(0x7f5c0809bc50 msgr2=0x7f5c080d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:58.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- 172.21.15.32:0/2494657487 shutdown_connections 2023-10-16T08:16:58.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- 172.21.15.32:0/2494657487 wait complete. 2023-10-16T08:16:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 Processor -- start 2023-10-16T08:16:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- start start 2023-10-16T08:16:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c080cf5d0 0x7f5c081532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c080d3550 0x7f5c0815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 0x7f5c081590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08154020 con 0x7f5c0815ca70 2023-10-16T08:16:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08154300 con 0x7f5c080d3550 2023-10-16T08:16:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c0e086640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c081545e0 con 0x7f5c080cf5d0 2023-10-16T08:16:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 0x7f5c081590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 0x7f5c081590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46004/0 (socket says 172.21.15.32:46004) 2023-10-16T08:16:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c07fff640 1 -- 172.21.15.32:0/4169660894 learned_addr learned my addr 172.21.15.32:0/4169660894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c077fe640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c080cf5d0 0x7f5c081532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.426+0000 7f5c06ffd640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c080d3550 0x7f5c0815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c07fff640 1 -- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c080cf5d0 msgr2=0x7f5c081532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c07fff640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c080cf5d0 0x7f5c081532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c07fff640 1 -- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c080d3550 msgr2=0x7f5c0815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c07fff640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c080d3550 0x7f5c0815fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c07fff640 1 -- 172.21.15.32:0/4169660894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc0094d0 con 0x7f5c0815ca70 2023-10-16T08:16:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c07fff640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 0x7f5c081590f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000c770 tx=0x7f5c0000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00003b80 con 0x7f5c0815ca70 2023-10-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00003ce0 con 0x7f5c0815ca70 2023-10-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c081548c0 con 0x7f5c0815ca70 2023-10-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000aa90 con 0x7f5c0815ca70 2023-10-16T08:16:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c08153820 con 0x7f5c0815ca70 2023-10-16T08:16:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c08074010 con 0x7f5c0815ca70 2023-10-16T08:16:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c0000abf0 con 0x7f5c0815ca70 2023-10-16T08:16:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5beffff640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5bf40664a0 0x7f5bf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.430+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5c00083f00 con 0x7f5c0815ca70 2023-10-16T08:16:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.434+0000 7f5c077fe640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5bf40664a0 0x7f5bf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.434+0000 7f5c077fe640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5bf40664a0 0x7f5bf4068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8006e30 tx=0x7f5bf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.434+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c000510a0 con 0x7f5c0815ca70 2023-10-16T08:16:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c08154d70 con 0x7f5c0815ca70 2023-10-16T08:16:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5beffff640 1 -- 172.21.15.32:0/4169660894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f5c000546c0 con 0x7f5c0815ca70 2023-10-16T08:16:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:16:58.631 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:16:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:16:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5bf40664a0 msgr2=0x7f5bf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5bf40664a0 0x7f5bf4068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8006e30 tx=0x7f5bf8008040 comp rx=0 tx=0).stop 2023-10-16T08:16:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 msgr2=0x7f5c081590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 0x7f5c081590f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000c770 tx=0x7f5c0000cc30 comp rx=0 tx=0).stop 2023-10-16T08:16:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 shutdown_connections 2023-10-16T08:16:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0815ca70 0x7f5c081590f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c080d3550 0x7f5c0815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5bf40664a0 0x7f5bf4068960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 --2- 172.21.15.32:0/4169660894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5c080cf5d0 0x7f5c081532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 >> 172.21.15.32:0/4169660894 conn(0x7f5c0809bc50 msgr2=0x7f5c0806bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 shutdown_connections 2023-10-16T08:16:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:58.626+0000 7f5c0e086640 1 -- 172.21.15.32:0/4169660894 wait complete. 2023-10-16T08:16:59.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T08:16:22.323436+0000', 'modified': '2023-10-16T08:16:46.160332+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4311': {'gid': 4311, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1428800105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1428800105}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1428800105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T08:16:59.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T08:16:59.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T08:16:59.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.194+0000 7f20ede8f640 1 Processor -- start 2023-10-16T08:16:59.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ede8f640 1 -- start start 2023-10-16T08:16:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ede8f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f20e815a6c0 0x7f20e80c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ede8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e806ffa0 0x7f20e815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ede8f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80c9cd0 con 0x7f20e80afd90 2023-10-16T08:16:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ede8f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e815e5b0 con 0x7f20e806ffa0 2023-10-16T08:16:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ede8f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e815aaa0 con 0x7f20e815a6c0 2023-10-16T08:16:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20e7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20e806ffa0 0x7f20e815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ed68e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f20e80afd90 0x7f20e80b1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35904/0 (socket says 172.21.15.32:35904) 2023-10-16T08:16:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ed68e640 1 -- 172.21.15.32:0/1777346215 learned_addr learned my addr 172.21.15.32:0/1777346215 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20e7fff640 1 -- 172.21.15.32:0/1777346215 >> v2:172.21.15.186:3300/0 conn(0x7f20e815a6c0 msgr2=0x7f20e80c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1777346215 >> v2:172.21.15.186:3300/0 conn(0x7f20e815a6c0 0x7f20e80c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1777346215 >> v2:172.21.15.186:3300/0 conn(0x7f20e815a6c0 0x7f20e80c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20e77fe640 1 -- 172.21.15.32:0/1777346215 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075617020 0 0) 0x7f20e00014a0 con 0x7f20e80afd90 2023-10-16T08:16:59.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20e7fff640 1 -- 172.21.15.32:0/1777346215 >> v1:172.21.15.32:6789/0 conn(0x7f20e80afd90 legacy=0x7f20e80b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20e7fff640 1 -- 172.21.15.32:0/1777346215 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e815e280 con 0x7f20e806ffa0 2023-10-16T08:16:59.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.198+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1777346215 >> v2:172.21.15.186:3300/0 conn(0x7f20e815a6c0 0x7f20e80c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:59.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1777346215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e806ffa0 0x7f20e815ea40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f20e80b09e0 tx=0x7f20d401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1fe23771fd3cd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20e77fe640 1 -- 172.21.15.32:0/1777346215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40a7070 con 0x7f20e806ffa0 2023-10-16T08:16:59.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20e77fe640 1 -- 172.21.15.32:0/1777346215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40ab430 con 0x7f20e806ffa0 2023-10-16T08:16:59.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20e77fe640 1 -- 172.21.15.32:0/1777346215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40ab7c0 con 0x7f20e806ffa0 2023-10-16T08:16:59.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 -- 172.21.15.32:0/1777346215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e806ffa0 msgr2=0x7f20e815ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1777346215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e806ffa0 0x7f20e815ea40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f20e80b09e0 tx=0x7f20d401c430 comp rx=0 tx=0).stop 2023-10-16T08:16:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 -- 172.21.15.32:0/1777346215 shutdown_connections 2023-10-16T08:16:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1777346215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e806ffa0 0x7f20e815ea40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1777346215 >> v2:172.21.15.186:3300/0 conn(0x7f20e815a6c0 0x7f20e80c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 -- 172.21.15.32:0/1777346215 >> 172.21.15.32:0/1777346215 conn(0x7f20e806a4f0 msgr2=0x7f20e80cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 -- 172.21.15.32:0/1777346215 shutdown_connections 2023-10-16T08:16:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 -- 172.21.15.32:0/1777346215 wait complete. 2023-10-16T08:16:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 Processor -- start 2023-10-16T08:16:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.202+0000 7f20ede8f640 1 -- start start 2023-10-16T08:16:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20e806ffa0 0x7f20e81f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20e80afd90 0x7f20e8156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 0x7f20e80c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80c8770 con 0x7f20e815a6c0 2023-10-16T08:16:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80c3a40 con 0x7f20e80afd90 2023-10-16T08:16:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80c6a50 con 0x7f20e806ffa0 2023-10-16T08:16:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 0x7f20e80c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 0x7f20e80c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46032/0 (socket says 172.21.15.32:46032) 2023-10-16T08:16:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 -- 172.21.15.32:0/1844094227 learned_addr learned my addr 172.21.15.32:0/1844094227 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20e80afd90 0x7f20e8156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20e806ffa0 0x7f20e81f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 -- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20e806ffa0 msgr2=0x7f20e81f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20e806ffa0 0x7f20e81f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 -- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20e80afd90 msgr2=0x7f20e8156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20e80afd90 0x7f20e8156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 -- 172.21.15.32:0/1844094227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4009420 con 0x7f20e815a6c0 2023-10-16T08:16:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20e806ffa0 0x7f20e81f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ed68e640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 0x7f20e80c7e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c810 tx=0x7f20e000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20e57fa640 1 -- 172.21.15.32:0/1844094227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000a640 con 0x7f20e815a6c0 2023-10-16T08:16:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20e57fa640 1 -- 172.21.15.32:0/1844094227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000a7a0 con 0x7f20e815a6c0 2023-10-16T08:16:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20e57fa640 1 -- 172.21.15.32:0/1844094227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000ab50 con 0x7f20e815a6c0 2023-10-16T08:16:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e80b0730 con 0x7f20e815a6c0 2023-10-16T08:16:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.206+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e80c33f0 con 0x7f20e815a6c0 2023-10-16T08:16:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e57fa640 1 -- 172.21.15.32:0/1844094227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f20e000acb0 con 0x7f20e815a6c0 2023-10-16T08:16:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e57fa640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f20cc0664a0 0x7f20cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e57fa640 1 -- 172.21.15.32:0/1844094227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f20e0083600 con 0x7f20e815a6c0 2023-10-16T08:16:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f20cc0664a0 0x7f20cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 Processor -- start 2023-10-16T08:16:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 -- start start 2023-10-16T08:16:59.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f20cc0664a0 0x7f20cc068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20e81f64a0 tx=0x7f20dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b5af0 0x56173d7b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d76bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d76c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d55ccc0 con 0x56173d7b6410 2023-10-16T08:16:59.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d5bf920 con 0x56173d7b5af0 2023-10-16T08:16:59.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56173d5bce50 con 0x56173d76c060 2023-10-16T08:16:59.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d76bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20ece8d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b5af0 0x56173d7b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20ece8d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b5af0 0x56173d7b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:34916/0 (socket says 172.21.15.32:34916) 2023-10-16T08:16:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20ece8d640 1 -- 172.21.15.32:0/2534290987 learned_addr learned my addr 172.21.15.32:0/2534290987 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20ed68e640 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d76c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e7fff640 1 -- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 msgr2=0x56173d76c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e7fff640 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d76c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e7fff640 1 -- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b5af0 msgr2=0x56173d7b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e7fff640 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b5af0 0x56173d7b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.210+0000 7f20e7fff640 1 -- 172.21.15.32:0/2534290987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173d2b1890 con 0x56173d7b6410 2023-10-16T08:16:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20e7fff640 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d76bb20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f20d40ac000 tx=0x7f20d4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2466c2011b4b43 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20d27fc640 1 -- 172.21.15.32:0/2534290987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40abb30 con 0x56173d7b6410 2023-10-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20d27fc640 1 -- 172.21.15.32:0/2534290987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40abcb0 con 0x56173d7b6410 2023-10-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20d27fc640 1 -- 172.21.15.32:0/2534290987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40bf690 con 0x56173d7b6410 2023-10-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- 172.21.15.32:0/2534290987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 msgr2=0x56173d76bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d76bb20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f20d40ac000 tx=0x7f20d4004740 comp rx=0 tx=0).stop 2023-10-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- 172.21.15.32:0/2534290987 shutdown_connections 2023-10-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d76c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d76bb20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- 172.21.15.32:0/2534290987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b5af0 0x56173d7b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- 172.21.15.32:0/2534290987 >> 172.21.15.32:0/2534290987 conn(0x56173d75efc0 msgr2=0x56173d765ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- 172.21.15.32:0/2534290987 shutdown_connections 2023-10-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- 172.21.15.32:0/2534290987 wait complete. 2023-10-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- start start 2023-10-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 Processor -- start 2023-10-16T08:16:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d76c060 0x56173d7a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d7a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d7a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d5b8740 con 0x56173d7b6410 2023-10-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d577d10 con 0x56173d76c060 2023-10-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56173d5779b0 con 0x56173d7b5af0 2023-10-16T08:16:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20f2464000 10 client.0 _ll_register_callbacks cb 0x56173d753390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20ed68e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d7a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20ed68e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d7a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46042/0 (socket says 172.21.15.32:46042) 2023-10-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20ed68e640 1 -- 172.21.15.32:0/2477287763 learned_addr learned my addr 172.21.15.32:0/2477287763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20e7fff640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d7a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.214+0000 7f20ece8d640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d76c060 0x56173d7a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ed68e640 1 -- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 msgr2=0x56173d7a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ed68e640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d7a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ed68e640 1 -- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d76c060 msgr2=0x56173d7a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ed68e640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d76c060 0x56173d7a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ed68e640 1 -- 172.21.15.32:0/2477287763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e815e280 con 0x56173d7b6410 2023-10-16T08:16:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ece8d640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d76c060 0x56173d7a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ed68e640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d7a3cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20e0081830 tx=0x7f20e0054440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa01eb021fe9414 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:16:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 1 -- 172.21.15.32:0/2477287763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e005c4b0 con 0x56173d7b6410 2023-10-16T08:16:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 1 -- 172.21.15.32:0/2477287763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f20e0069070 con 0x56173d7b6410 2023-10-16T08:16:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56173d56c370 con 0x56173d7b6410 2023-10-16T08:16:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 1 -- 172.21.15.32:0/2477287763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e00580b0 con 0x56173d7b6410 2023-10-16T08:16:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 10 client.4857 fetch_fsmap learned FSMap version 9 2023-10-16T08:16:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56173d57a460 con 0x56173d7b6410 2023-10-16T08:16:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 1 -- 172.21.15.32:0/2477287763 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173d56c370 con 0x56173d7b6410 2023-10-16T08:16:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 1 -- 172.21.15.32:0/2477287763 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f20e0058210 con 0x56173d7b6410 2023-10-16T08:16:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2023-10-16T08:16:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 10 client.4857 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T08:16:59.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 20 client.4857 populate_metadata read hostname 'smithi032' 2023-10-16T08:16:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d7c4b80 0x56173d7c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 4 client.4857 mds_command: new command op to 4311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T08:16:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 --> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56173d5522d0 con 0x56173d7c4b80 2023-10-16T08:16:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ece8d640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d7c4b80 0x56173d7c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 10 client.4857 ms_handle_connect on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20ece8d640 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d7c4b80 0x56173d7c6f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93781ff11946f99 server_cookie=479fb5c76e30f23c in_seq=0 out_seq=0 2023-10-16T08:16:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 1 -- 172.21.15.32:0/2477287763 <== mds.0 v2:172.21.15.32:6834/1428800105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56173d5522d0 con 0x56173d7c4b80 2023-10-16T08:16:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 10 client.4857 handle_command_reply: tid=0 2023-10-16T08:16:59.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 client.4857 shutdown 2023-10-16T08:16:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 msgr2=0x56173d7a3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d7a3cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20e0081830 tx=0x7f20e0054440 comp rx=0 tx=0).stop 2023-10-16T08:16:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 shutdown_connections 2023-10-16T08:16:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d7c4b80 0x56173d7c6f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b6410 0x56173d7a3cb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d7a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20c77fe640 0 client.4857 ms_handle_reset on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 --2- 172.21.15.32:0/2477287763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d76c060 0x56173d7a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 >> 172.21.15.32:0/2477287763 conn(0x56173d75efc0 msgr2=0x56173d765760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 shutdown_connections 2023-10-16T08:16:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 1 -- 172.21.15.32:0/2477287763 wait complete. 2023-10-16T08:16:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.218+0000 7f20f2464000 20 client.4857 trim_cache size 0 max 16384 2023-10-16T08:16:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 Processor -- start 2023-10-16T08:16:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 -- start start 2023-10-16T08:16:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 0x56173d810730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d810c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d8111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d272940 con 0x56173d76c060 2023-10-16T08:16:59.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d58ea10 con 0x56173d7b6410 2023-10-16T08:16:59.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56173d2b21b0 con 0x56173d7b5af0 2023-10-16T08:16:59.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 0x56173d810730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 0x56173d810730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46056/0 (socket says 172.21.15.32:46056) 2023-10-16T08:16:59.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 -- 172.21.15.32:0/1272881650 learned_addr learned my addr 172.21.15.32:0/1272881650 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:59.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d810c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ed68e640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d8111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 -- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 msgr2=0x56173d810c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d810c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 -- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 msgr2=0x56173d8111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d8111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ece8d640 1 -- 172.21.15.32:0/1272881650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173d560760 con 0x56173d76c060 2023-10-16T08:16:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d810c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:59.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.234+0000 7f20ed68e640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d8111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 0x56173d810730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x56173d5b2ab0 tx=0x7f20dc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865d9bf23dde3f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20d2ffd640 1 -- 172.21.15.32:0/1272881650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc026070 con 0x56173d76c060 2023-10-16T08:16:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20d2ffd640 1 -- 172.21.15.32:0/1272881650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00c040 con 0x56173d76c060 2023-10-16T08:16:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20d2ffd640 1 -- 172.21.15.32:0/1272881650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00b050 con 0x56173d76c060 2023-10-16T08:16:59.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- 172.21.15.32:0/1272881650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 msgr2=0x56173d810730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 0x56173d810730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x56173d5b2ab0 tx=0x7f20dc013000 comp rx=0 tx=0).stop 2023-10-16T08:16:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- 172.21.15.32:0/1272881650 shutdown_connections 2023-10-16T08:16:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d8111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d7b5af0 0x56173d810c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- 172.21.15.32:0/1272881650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d76c060 0x56173d810730 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- 172.21.15.32:0/1272881650 >> 172.21.15.32:0/1272881650 conn(0x56173d75efc0 msgr2=0x56173d7a1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- 172.21.15.32:0/1272881650 shutdown_connections 2023-10-16T08:16:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- 172.21.15.32:0/1272881650 wait complete. 2023-10-16T08:16:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- start start 2023-10-16T08:16:59.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 Processor -- start 2023-10-16T08:16:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d7c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 0x56173d7d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d7d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d774a30 con 0x56173d7b5af0 2023-10-16T08:16:59.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173d7b4ca0 con 0x56173d7b6410 2023-10-16T08:16:59.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x56173d5b5f50 con 0x56173d76c060 2023-10-16T08:16:59.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20f2464000 10 client.0 _ll_register_callbacks cb 0x7f20d4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:16:59.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 0x56173d7d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 0x56173d7d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46072/0 (socket says 172.21.15.32:46072) 2023-10-16T08:16:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 -- 172.21.15.32:0/1997837718 learned_addr learned my addr 172.21.15.32:0/1997837718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d7c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20ed68e640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d7d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 -- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 msgr2=0x56173d7c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d7c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 -- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 msgr2=0x56173d7d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d7d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 -- 172.21.15.32:0/1997837718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc00dd80 con 0x56173d7b5af0 2023-10-16T08:16:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20e7fff640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 0x56173d7d0330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20d4008010 tx=0x7f20d40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914d5420126911d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.238+0000 7f20c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:16:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 1 -- 172.21.15.32:0/1997837718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40bf450 con 0x56173d7b5af0 2023-10-16T08:16:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 1 -- 172.21.15.32:0/1997837718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f20d40c2a50 con 0x56173d7b5af0 2023-10-16T08:16:59.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56173d5506f0 con 0x56173d7b5af0 2023-10-16T08:16:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 1 -- 172.21.15.32:0/1997837718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d40c2c10 con 0x56173d7b5af0 2023-10-16T08:16:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 10 client.4863 fetch_fsmap learned FSMap version 9 2023-10-16T08:16:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56173d2b1cc0 con 0x56173d7b5af0 2023-10-16T08:16:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 1 -- 172.21.15.32:0/1997837718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40d2020 con 0x56173d7b5af0 2023-10-16T08:16:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 1 -- 172.21.15.32:0/1997837718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f20d40c2100 con 0x56173d7b5af0 2023-10-16T08:16:59.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2023-10-16T08:16:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 10 client.4863 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T08:16:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 20 client.4863 populate_metadata read hostname 'smithi032' 2023-10-16T08:16:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d814b60 0x56173d814f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 4 client.4863 mds_command: new command op to 4311 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T08:16:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 --> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56173d5b1720 con 0x56173d814b60 2023-10-16T08:16:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d814b60 0x56173d814f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 10 client.4863 ms_handle_connect on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ece8d640 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d814b60 0x56173d814f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22f31cfa8245871 server_cookie=1c785389cd8af4f in_seq=0 out_seq=0 2023-10-16T08:16:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 1 -- 172.21.15.32:0/1997837718 <== mds.0 v2:172.21.15.32:6834/1428800105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56173d5b1720 con 0x56173d814b60 2023-10-16T08:16:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 10 client.4863 handle_command_reply: tid=0 2023-10-16T08:16:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 client.4863 shutdown 2023-10-16T08:16:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 msgr2=0x56173d7d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 0x56173d7d0330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20d4008010 tx=0x7f20d40046d0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 shutdown_connections 2023-10-16T08:16:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:6834/1428800105,v1:172.21.15.32:6835/1428800105] conn(0x56173d814b60 0x56173d814f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56173d7b6410 0x56173d7d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56173d7b5af0 0x56173d7d0330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 --2- 172.21.15.32:0/1997837718 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x56173d76c060 0x56173d7c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 >> 172.21.15.32:0/1997837718 conn(0x56173d75efc0 msgr2=0x56173d7a22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20c77fe640 0 client.4863 ms_handle_reset on v2:172.21.15.32:6834/1428800105 2023-10-16T08:16:59.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 shutdown_connections 2023-10-16T08:16:59.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 1 -- 172.21.15.32:0/1997837718 wait complete. 2023-10-16T08:16:59.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20f2464000 20 client.4863 trim_cache size 0 max 16384 2023-10-16T08:16:59.271 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:16:59.271 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "943c836a-5c4c-4cb7-a190-9643b3476f99", 2023-10-16T08:16:59.271 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-10-16T08:16:59.271 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4311, 2023-10-16T08:16:59.271 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-10-16T08:16:59.272 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-10-16T08:16:59.272 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-10-16T08:16:59.272 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 36.288286562000003, 2023-10-16T08:16:59.272 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 9, 2023-10-16T08:16:59.272 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 25, 2023-10-16T08:16:59.273 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 25, 2023-10-16T08:16:59.273 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 43.243586022000002 2023-10-16T08:16:59.273 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:16:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f20cc0664a0 msgr2=0x7f20cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f20cc0664a0 0x7f20cc068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20e81f64a0 tx=0x7f20dc008040 comp rx=0 tx=0).stop 2023-10-16T08:16:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 msgr2=0x7f20e80c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 0x7f20e80c7e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c810 tx=0x7f20e000ccd0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 shutdown_connections 2023-10-16T08:16:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e815a6c0 0x7f20e80c7e90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20e80afd90 0x7f20e8156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f20cc0664a0 0x7f20cc068960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 --2- 172.21.15.32:0/1844094227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f20e806ffa0 0x7f20e81f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 >> 172.21.15.32:0/1844094227 conn(0x7f20e806a4f0 msgr2=0x7f20e80af660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 shutdown_connections 2023-10-16T08:16:59.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.242+0000 7f20ede8f640 1 -- 172.21.15.32:0/1844094227 wait complete. 2023-10-16T08:16:59.277 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-16T08:16:59.288 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-16T08:16:59.289 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-16T08:16:59.289 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-16T08:16:59.290 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:16:59.290 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:16:59.290 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-16T08:16:59.290 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:16:59.291 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T08:16:59.291 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-16T08:16:59.291 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:16:59.291 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T08:16:59.292 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-16T08:16:59.292 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:16:59.292 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T08:16:59.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:59.293 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-10-16T08:16:59.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:59.305 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-10-16T08:16:59.371 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2023-10-16T08:16:59.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:16:59.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:59.374 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:16:59.384 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-16T08:16:59.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:59.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T08:16:59.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:16:59.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-10-16T08:16:59.645 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-16T08:16:59.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:16:59.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.782+0000 7fd8d5046640 1 Processor -- start 2023-10-16T08:16:59.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.782+0000 7fd8d5046640 1 -- start start 2023-10-16T08:16:59.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8d5046640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8d015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8d5046640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd8d00b4020 0x7fd8d00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8d5046640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d015be60 con 0x7fd8d00b0620 2023-10-16T08:16:59.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8d5046640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d015b3b0 con 0x7fd8d00b4020 2023-10-16T08:16:59.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8d5046640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00b0a90 con 0x7fd8d0070c40 2023-10-16T08:16:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cf59e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd8d00b0620 0x7fd8d015c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35912/0 (socket says 172.21.15.32:35912) 2023-10-16T08:16:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cf59e640 1 -- 172.21.15.32:0/3071244592 learned_addr learned my addr 172.21.15.32:0/3071244592 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:16:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266801962 0 0) 0x7fd8d015be60 con 0x7fd8d00b0620 2023-10-16T08:16:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b8003530 con 0x7fd8d00b0620 2023-10-16T08:16:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/3071244592 >> v2:172.21.15.186:3300/0 conn(0x7fd8d00b4020 0x7fd8d00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8d015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513616720 0 0) 0x7fd8b8003530 con 0x7fd8d00b0620 2023-10-16T08:16:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 msgr2=0x7fd8d015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8d015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 >> v2:172.21.15.186:3300/0 conn(0x7fd8d00b4020 msgr2=0x7fd8d00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 --2- 172.21.15.32:0/3071244592 >> v2:172.21.15.186:3300/0 conn(0x7fd8d00b4020 0x7fd8d00b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d015c1f0 con 0x7fd8d00b0620 2023-10-16T08:16:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fd8c8001a00 con 0x7fd8d00b0620 2023-10-16T08:16:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 >> v1:172.21.15.32:6789/0 conn(0x7fd8d00b0620 legacy=0x7fd8d015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b8004d10 0x7fd8b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d015b3b0 con 0x7fd8b8003980 2023-10-16T08:16:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00b0a90 con 0x7fd8b8004d10 2023-10-16T08:16:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b800b990 con 0x7fd8d0070c40 2023-10-16T08:16:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8cf59e640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b8004d10 0x7fd8b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 -- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 msgr2=0x7fd8b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 -- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b8004d10 msgr2=0x7fd8b8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:16:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8b8004d10 0x7fd8b8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ced9d640 1 -- 172.21.15.32:0/3071244592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8003530 con 0x7fd8b8003980 2023-10-16T08:16:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.786+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d0070c40 0x7fd8b80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:16:59.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8b8003da0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009ae0 tx=0x7fd8c001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06ce7d75a215f04 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00a5530 con 0x7fd8b8003980 2023-10-16T08:16:59.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c00a5690 con 0x7fd8b8003980 2023-10-16T08:16:59.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8cdd9b640 1 -- 172.21.15.32:0/3071244592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00a5a40 con 0x7fd8b8003980 2023-10-16T08:16:59.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8d5046640 1 -- 172.21.15.32:0/3071244592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 msgr2=0x7fd8b8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8d5046640 1 --2- 172.21.15.32:0/3071244592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8b8003da0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009ae0 tx=0x7fd8c001ca50 comp rx=0 tx=0).stop 2023-10-16T08:16:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8cf59e640 1 -- 172.21.15.32:0/3071244592 reap_dead start 2023-10-16T08:16:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8d5046640 1 -- 172.21.15.32:0/3071244592 shutdown_connections 2023-10-16T08:16:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8d5046640 1 -- 172.21.15.32:0/3071244592 >> 172.21.15.32:0/3071244592 conn(0x7fd8d006c190 msgr2=0x7fd8d006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:16:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8d5046640 1 -- 172.21.15.32:0/3071244592 shutdown_connections 2023-10-16T08:16:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.790+0000 7fd8d5046640 1 -- 172.21.15.32:0/3071244592 wait complete. 2023-10-16T08:16:59.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 Processor -- start 2023-10-16T08:16:59.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- start start 2023-10-16T08:16:59.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8d0154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d00c4a60 0x7fd8d00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8d00cd7c0 0x7fd8d00a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00cdc40 con 0x7fd8b8003980 2023-10-16T08:16:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00a8220 con 0x7fd8d00cd7c0 2023-10-16T08:16:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c3770 con 0x7fd8d00c4a60 2023-10-16T08:16:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8d0154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8d0154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46096/0 (socket says 172.21.15.32:46096) 2023-10-16T08:16:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 -- 172.21.15.32:0/2552662525 learned_addr learned my addr 172.21.15.32:0/2552662525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:16:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d00c4a60 0x7fd8d00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 -- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d00c4a60 msgr2=0x7fd8d00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8cf59e640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8d00cd7c0 0x7fd8d00a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d00c4a60 0x7fd8d00cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 -- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8d00cd7c0 msgr2=0x7fd8d00a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:16:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8d00cd7c0 0x7fd8d00a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:16:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 -- 172.21.15.32:0/2552662525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00c45d0 con 0x7fd8b8003980 2023-10-16T08:16:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d00c4a60 0x7fd8d00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8cf59e640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8d00cd7c0 0x7fd8d00a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:16:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8ced9d640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8d0154700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009ae0 tx=0x7fd8b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00bc070 con 0x7fd8b8003980 2023-10-16T08:16:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c00a5e80 con 0x7fd8b8003980 2023-10-16T08:16:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0004020 con 0x7fd8b8003980 2023-10-16T08:16:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00c39c0 con 0x7fd8b8003980 2023-10-16T08:16:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d00c3f80 con 0x7fd8b8003980 2023-10-16T08:16:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.794+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d0074010 con 0x7fd8b8003980 2023-10-16T08:16:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.798+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd8c00b4040 con 0x7fd8b8003980 2023-10-16T08:16:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.798+0000 7fd8b77fe640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8bc0664a0 0x7fd8bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:16:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.798+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8c011d8c0 con 0x7fd8b8003980 2023-10-16T08:16:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.798+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8bc0664a0 0x7fd8bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:16:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.798+0000 7fd8ce59c640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8bc0664a0 0x7fd8bc068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00acbc0 tx=0x7fd8c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:16:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.802+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd8c00eaa40 con 0x7fd8b8003980 2023-10-16T08:16:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.990+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd8d00ce220 con 0x7fd8b8003980 2023-10-16T08:16:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.990+0000 7fd8b77fe640 1 -- 172.21.15.32:0/2552662525 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd8c00ee060 con 0x7fd8b8003980 2023-10-16T08:16:59.998 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T08:17:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8bc0664a0 msgr2=0x7fd8bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8bc0664a0 0x7fd8bc068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00acbc0 tx=0x7fd8c4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 msgr2=0x7fd8d0154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8d0154700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009ae0 tx=0x7fd8b8005800 comp rx=0 tx=0).stop 2023-10-16T08:17:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 shutdown_connections 2023-10-16T08:17:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8d00cd7c0 0x7fd8d00a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8bc0664a0 0x7fd8bc068960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd8d00c4a60 0x7fd8d00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 --2- 172.21.15.32:0/2552662525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8003980 0x7fd8d0154700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 >> 172.21.15.32:0/2552662525 conn(0x7fd8d006c190 msgr2=0x7fd8d00adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 shutdown_connections 2023-10-16T08:17:00.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:16:59.994+0000 7fd8d5046640 1 -- 172.21.15.32:0/2552662525 wait complete. 2023-10-16T08:17:00.019 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:17:00.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:17:00.020 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:17:00.020 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:17:00.020 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:00.021 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:17:00.021 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:17:00.021 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:17:00.021 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:17:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.022 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:17:00.032 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:00.032 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:00.033 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:00.033 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:00.033 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:00.033 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:00.033 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:00.034 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:00.034 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:17:00.034 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:00.034 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:00.035 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:17:00.035 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:17:00.035 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:00.035 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:17:00.035 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:17:00.035 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42385sec preferred_lft 42385sec 2023-10-16T08:17:00.036 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:17:00.036 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:00.036 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:00.036 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:00.036 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:17:00.037 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:17:00.037 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:00.037 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:17:00.037 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:00.038 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T08:17:00.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.038 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:00.039 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:00.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-10-16T08:17:00.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-10-16T08:17:00.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-10-16T08:17:00.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T08:17:00.040 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:00.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.147 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:17:00.166 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:17:00.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.169 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:17:00.219 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:17:00.221 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:17:00.221 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:17:00.222 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:17:00.222 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:17:00.222 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:17:00.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.223 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:00.223 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:00.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:17:00.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:17:00.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:17:00.225 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:00.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.318 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:17:00.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.328 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-10-16T08:17:00.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.383 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:00.383 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:00.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:00.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-16T08:17:00.384 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:00.466 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-16T08:17:00.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.467 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:00.467 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:00.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-16T08:17:00.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T08:17:00.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-16T08:17:00.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-16T08:17:00.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-16T08:17:00.469 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:00.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.608 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:00.608 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:00.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:17:00.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T08:17:00.609 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:00.649 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:17:00.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:00.650 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:00.698 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:17:00.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:00.699 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:00.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:17:00.819 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:00.871 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:17:00.872 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:17:00.872 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:17:00.872 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:17:00.872 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:17:00.872 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:17:00.873 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:17:00.873 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:17:00.873 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:17:00.873 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:17:00.873 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:17:00.874 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:17:00.874 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:00.874 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:17:00.874 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:17:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:17:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:17:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:17:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:17:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:17:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:17:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:17:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:17:00.877 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:17:00.877 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:17:00.877 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:17:00.877 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:17:00.878 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:00.878 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:00.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:00.879 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:17:00.929 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:17:00.930 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:17:00.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:17:00.999 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:01.055 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:17:01.056 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:17:01.056 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:17:01.056 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:17:01.056 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:17:01.056 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:17:01.057 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:17:01.057 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:17:01.057 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:17:01.057 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:17:01.057 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:17:01.058 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:17:01.058 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:01.058 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:17:01.058 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:17:01.058 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:17:01.059 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:17:01.059 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:17:01.059 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:01.059 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:17:01.060 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:01.060 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:17:01.060 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:17:01.060 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:17:01.060 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:17:01.060 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:17:01.061 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:17:01.061 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:17:01.061 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:17:01.061 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:01.061 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:01.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:01.063 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:17:01.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:17:01.070+0000 7fde7c931580 -1 init, newargv = 0x5635b029edd0 newargc=15 2023-10-16T08:17:01.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[44814]: starting ceph client 2023-10-16T08:17:01.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[44814]: starting fuse 2023-10-16T08:17:01.084 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:17:01.085 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:17:01.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:17:01.085 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:17:01.086 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:17:01.086 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:17:01.086 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:17:01.087 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:01.087 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:17:01.087 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:17:01.088 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:17:01.088 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:17:01.088 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:01.088 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:17:01.089 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:17:01.089 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:17:01.089 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:17:01.090 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:01.090 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:17:01.090 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:17:01.091 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:17:01.091 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:17:01.091 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:17:01.091 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:17:01.092 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:17:01.092 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:17:01.092 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:01.093 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:17:01.093 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:17:01.213 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.44814.asok 2023-10-16T08:17:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:17:01.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44814.asok 2023-10-16T08:17:01.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:01.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44814.asok status 2023-10-16T08:17:01.397 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:17:01.398 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:17:01.398 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:17:01.398 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:17:01.399 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:17:01.399 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:17:01.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:17:01.399 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "44814", 2023-10-16T08:17:01.399 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:17:01.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:01.400 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:17:01.400 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:17:01.400 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4875, 2023-10-16T08:17:01.400 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:17:01.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:17:01.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:17:01.401 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 4875 2023-10-16T08:17:01.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:01.401 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:17:01.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:17:01.402 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:17:01.402 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3263819507 2023-10-16T08:17:01.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:01.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:01.403 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:17:01.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:17:01.403 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:17:01.403 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3263819507 2023-10-16T08:17:01.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:01.404 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.4875 192.168.0.1:0/3263819507", 2023-10-16T08:17:01.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3263819507", 2023-10-16T08:17:01.404 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:17:01.404 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 9, 2023-10-16T08:17:01.404 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 25, 2023-10-16T08:17:01.405 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:17:01.405 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:17:01.405 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2023-10-16T08:17:01.405 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:17:01.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:17:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.550+0000 7f61e8a91640 1 Processor -- start 2023-10-16T08:17:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.550+0000 7f61e8a91640 1 -- start start 2023-10-16T08:17:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e8a91640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61e41597c0 0x7f61e40ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e8a91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 0x7f61e40cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e8a91640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e40723d0 con 0x7f61e40adc10 2023-10-16T08:17:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e8a91640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e40717c0 con 0x7f61e41597c0 2023-10-16T08:17:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e8a91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4071aa0 con 0x7f61e4159ba0 2023-10-16T08:17:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 0x7f61e40cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e259c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f61e41597c0 0x7f61e40ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 0x7f61e40cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:42658/0 (socket says 172.21.15.32:42658) 2023-10-16T08:17:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 -- 172.21.15.32:0/275519475 learned_addr learned my addr 172.21.15.32:0/275519475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 -- 172.21.15.32:0/275519475 >> v2:172.21.15.186:3300/0 conn(0x7f61e41597c0 msgr2=0x7f61e40ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 --2- 172.21.15.32:0/275519475 >> v2:172.21.15.186:3300/0 conn(0x7f61e41597c0 0x7f61e40ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 -- 172.21.15.32:0/275519475 >> v1:172.21.15.186:6789/0 conn(0x7f61e40adc10 legacy=0x7f61e40cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61dbfff640 1 -- 172.21.15.32:0/275519475 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4071d80 con 0x7f61e4159ba0 2023-10-16T08:17:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.554+0000 7f61e259c640 1 --2- 172.21.15.32:0/275519475 >> v2:172.21.15.186:3300/0 conn(0x7f61e41597c0 0x7f61e40ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61dbfff640 1 --2- 172.21.15.32:0/275519475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 0x7f61e40cae10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009870 tx=0x7f61cc01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9c853a116be3e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e1d9b640 1 -- 172.21.15.32:0/275519475 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0aa640 con 0x7f61e4159ba0 2023-10-16T08:17:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e1d9b640 1 -- 172.21.15.32:0/275519475 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc0aa7a0 con 0x7f61e4159ba0 2023-10-16T08:17:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e1d9b640 1 -- 172.21.15.32:0/275519475 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0aab30 con 0x7f61e4159ba0 2023-10-16T08:17:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 -- 172.21.15.32:0/275519475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 msgr2=0x7f61e40cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 --2- 172.21.15.32:0/275519475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 0x7f61e40cae10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009870 tx=0x7f61cc01c540 comp rx=0 tx=0).stop 2023-10-16T08:17:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 -- 172.21.15.32:0/275519475 shutdown_connections 2023-10-16T08:17:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 --2- 172.21.15.32:0/275519475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e4159ba0 0x7f61e40cae10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 --2- 172.21.15.32:0/275519475 >> v2:172.21.15.186:3300/0 conn(0x7f61e41597c0 0x7f61e40ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 -- 172.21.15.32:0/275519475 >> 172.21.15.32:0/275519475 conn(0x7f61e406c190 msgr2=0x7f61e406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 -- 172.21.15.32:0/275519475 shutdown_connections 2023-10-16T08:17:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 -- 172.21.15.32:0/275519475 wait complete. 2023-10-16T08:17:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 Processor -- start 2023-10-16T08:17:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.558+0000 7f61e8a91640 1 -- start start 2023-10-16T08:17:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e40adc10 0x7f61e40c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 0x7f61e40c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e4159ba0 0x7f61e415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40c4810 con 0x7f61e41597c0 2023-10-16T08:17:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40c77e0 con 0x7f61e4159ba0 2023-10-16T08:17:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40c81b0 con 0x7f61e40adc10 2023-10-16T08:17:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 0x7f61e40c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 0x7f61e40c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46104/0 (socket says 172.21.15.32:46104) 2023-10-16T08:17:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 -- 172.21.15.32:0/649328369 learned_addr learned my addr 172.21.15.32:0/649328369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e259c640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e40adc10 0x7f61e40c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e2d9d640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e4159ba0 0x7f61e415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 -- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e40adc10 msgr2=0x7f61e40c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e40adc10 0x7f61e40c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 -- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e4159ba0 msgr2=0x7f61e415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e4159ba0 0x7f61e415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 -- 172.21.15.32:0/649328369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4009420 con 0x7f61e41597c0 2023-10-16T08:17:01.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61dbfff640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 0x7f61e40c7c70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61cc01a170 tx=0x7f61cc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0bd5f0 con 0x7f61e41597c0 2023-10-16T08:17:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc0bd770 con 0x7f61e41597c0 2023-10-16T08:17:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc0094d0 con 0x7f61e41597c0 2023-10-16T08:17:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0bdb20 con 0x7f61e41597c0 2023-10-16T08:17:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e40c6bd0 con 0x7f61e41597c0 2023-10-16T08:17:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.562+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e40c7950 con 0x7f61e41597c0 2023-10-16T08:17:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.566+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61cc0a8050 con 0x7f61e41597c0 2023-10-16T08:17:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.566+0000 7f61daffd640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f61c40664a0 0x7f61c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:01.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.566+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61cc11de50 con 0x7f61e41597c0 2023-10-16T08:17:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.566+0000 7f61e259c640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f61c40664a0 0x7f61c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.566+0000 7f61e259c640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f61c40664a0 0x7f61c4068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61e414fd90 tx=0x7f61d4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.570+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61cc0eb190 con 0x7f61e41597c0 2023-10-16T08:17:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61e4074010 con 0x7f61e41597c0 2023-10-16T08:17:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61daffd640 1 -- 172.21.15.32:0/649328369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f61cc0aa600 con 0x7f61e41597c0 2023-10-16T08:17:01.768 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T08:17:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f61c40664a0 msgr2=0x7f61c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f61c40664a0 0x7f61c4068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61e414fd90 tx=0x7f61d4009190 comp rx=0 tx=0).stop 2023-10-16T08:17:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 msgr2=0x7f61e40c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 0x7f61e40c7c70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61cc01a170 tx=0x7f61cc01ad80 comp rx=0 tx=0).stop 2023-10-16T08:17:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 shutdown_connections 2023-10-16T08:17:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e4159ba0 0x7f61e415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e41597c0 0x7f61e40c7c70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f61c40664a0 0x7f61c4068960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 --2- 172.21.15.32:0/649328369 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61e40adc10 0x7f61e40c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 >> 172.21.15.32:0/649328369 conn(0x7f61e406c190 msgr2=0x7f61e40c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 shutdown_connections 2023-10-16T08:17:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:01.762+0000 7f61e8a91640 1 -- 172.21.15.32:0/649328369 wait complete. 2023-10-16T08:17:01.788 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:17:01.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:17:01.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:17:01.789 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-16T08:17:01.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:01.790 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:17:01.790 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:17:01.790 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:17:01.791 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-16T08:17:01.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:01.791 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:17:01.802 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:01.802 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.803 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:01.803 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:17:01.804 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:17:01.804 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:17:01.804 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:17:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:17:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42383sec preferred_lft 42383sec 2023-10-16T08:17:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:17:01.805 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.806 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:17:01.806 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:17:01.806 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:17:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.807 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:01.807 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:17:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-10-16T08:17:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.808 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:17:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:01.808 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-10-16T08:17:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:01.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:01.809 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:17:01.869 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:17:01.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:01.871 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:17:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:17:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:17:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:17:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:17:01.924 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:17:01.924 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:17:01.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:01.925 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:01.925 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:01.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:17:01.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:17:01.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:17:01.926 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:02.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.023 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:17:02.033 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:17:02.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.034 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-10-16T08:17:02.086 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-16T08:17:02.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.087 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:02.087 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:02.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:02.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-16T08:17:02.088 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:02.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T08:17:02.239 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:02.239 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:02.240 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:02.240 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.240 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:02.240 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.240 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:02.241 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 6a:c2:b4:2a:a1:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:17:02.241 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:17:02.241 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.241 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::68c2:b4ff:fe2a:a150/64 scope link tentative 2023-10-16T08:17:02.241 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T08:17:02.307 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:02.307 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:02.307 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:02.307 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.308 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:02.308 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.308 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:02.308 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 6a:c2:b4:2a:a1:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:17:02.308 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:17:02.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.309 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::68c2:b4ff:fe2a:a150/64 scope link tentative 2023-10-16T08:17:02.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:02.310 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-16T08:17:02.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.311 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:02.311 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:02.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-16T08:17:02.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T08:17:02.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-16T08:17:02.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-16T08:17:02.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-16T08:17:02.313 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:02.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.456 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:02.456 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:02.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2023-10-16T08:17:02.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T08:17:02.457 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:02.498 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-16T08:17:02.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:02.499 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:02.554 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-16T08:17:02.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:02.555 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:02.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:17:02.671 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:02.723 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:17:02.724 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:17:02.724 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:17:02.724 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:17:02.724 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:17:02.725 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:17:02.725 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:17:02.725 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:17:02.725 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:17:02.725 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:17:02.726 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:17:02.726 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:17:02.726 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:02.726 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:17:02.726 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:17:02.726 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:17:02.727 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:17:02.727 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:17:02.727 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:02.727 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:17:02.727 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:02.728 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:17:02.728 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:17:02.728 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:17:02.728 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:17:02.729 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:17:02.729 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:17:02.729 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:17:02.729 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:17:02.729 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:02.730 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:02.730 INFO:teuthology.orchestra.run.smithi032.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:02.730 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:02.730 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:02.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.732 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:17:02.782 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:17:02.782 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-10-16T08:17:02.783 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-16T08:17:02.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:17:02.851 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:02.903 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:17:02.904 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:17:02.904 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:17:02.904 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:17:02.904 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:17:02.905 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:17:02.905 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:17:02.905 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:17:02.905 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:17:02.905 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:17:02.906 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:17:02.906 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:17:02.906 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:02.906 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:17:02.906 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:17:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:17:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:17:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:17:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:17:02.907 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:02.908 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:17:02.908 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:17:02.908 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:17:02.908 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:17:02.908 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:17:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:17:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:17:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:17:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:02.910 INFO:teuthology.orchestra.run.smithi032.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:02.910 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:02.910 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:02.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:02.911 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:17:02.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi032.stderr:2023-10-16T08:17:02.926+0000 7f7ecd30a580 -1 init, newargv = 0x55fb7fc6da30 newargc=15 2023-10-16T08:17:02.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi032.stderr:ceph-fuse[44955]: starting ceph client 2023-10-16T08:17:02.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi032.stderr:ceph-fuse[44955]: starting fuse 2023-10-16T08:17:02.943 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:17:02.943 INFO:teuthology.orchestra.run.smithi032.stdout:44 2023-10-16T08:17:02.943 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-10-16T08:17:02.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:17:02.944 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:17:02.944 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:17:02.945 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:17:02.945 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:17:02.945 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:02.946 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:17:02.946 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-16T08:17:02.946 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:17:02.946 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-16T08:17:02.947 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:02.947 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:17:02.947 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:17:02.948 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:17:02.948 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:17:02.948 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:02.948 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:17:02.949 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:17:02.949 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:17:02.949 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:17:02.950 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:17:02.950 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:17:02.950 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:17:02.950 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:17:02.951 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:02.951 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.2")) 2023-10-16T08:17:02.951 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:17:03.069 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.2.44955.asok 2023-10-16T08:17:03.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:17:03.080 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.44955.asok 2023-10-16T08:17:03.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.44955.asok status 2023-10-16T08:17:03.258 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:17:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:17:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:17:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:17:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "2", 2023-10-16T08:17:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:17:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-16T08:17:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "44955", 2023-10-16T08:17:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:17:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:17:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:17:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4887, 2023-10-16T08:17:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:17:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:17:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:17:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 4887 2023-10-16T08:17:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:17:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:17:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:17:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3624712402 2023-10-16T08:17:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:17:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:17:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:17:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3624712402 2023-10-16T08:17:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.4887 192.168.0.2:0/3624712402", 2023-10-16T08:17:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.2:0/3624712402", 2023-10-16T08:17:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:17:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 9, 2023-10-16T08:17:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 26, 2023-10-16T08:17:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:17:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:17:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2023-10-16T08:17:03.265 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:17:03.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:17:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f5906b72640 1 Processor -- start 2023-10-16T08:17:03.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f5906b72640 1 -- start start 2023-10-16T08:17:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f5906b72640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f59001597c0 0x7f59000ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f5906b72640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000723d0 con 0x7f5900159ba0 2023-10-16T08:17:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f5906b72640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000717c0 con 0x7f59000adc10 2023-10-16T08:17:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f5906b72640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900071aa0 con 0x7f59001597c0 2023-10-16T08:17:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f58fffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5900159ba0 0x7f59000cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37212/0 (socket says 172.21.15.32:37212) 2023-10-16T08:17:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.410+0000 7f58fffff640 1 -- 172.21.15.32:0/1223460728 learned_addr learned my addr 172.21.15.32:0/1223460728 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590490e640 1 --2- 172.21.15.32:0/1223460728 >> v2:172.21.15.186:3300/0 conn(0x7f59001597c0 0x7f59000ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978866629 0 0) 0x7f59000723d0 con 0x7f5900159ba0 2023-10-16T08:17:03.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f0003530 con 0x7f5900159ba0 2023-10-16T08:17:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1686540170 0 0) 0x7f58f0003530 con 0x7f5900159ba0 2023-10-16T08:17:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 >> v2:172.21.15.186:3300/0 conn(0x7f59001597c0 msgr2=0x7f59000ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 --2- 172.21.15.32:0/1223460728 >> v2:172.21.15.186:3300/0 conn(0x7f59001597c0 0x7f59000ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 >> v1:172.21.15.186:6789/0 conn(0x7f59000adc10 legacy=0x7f590015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900071d80 con 0x7f5900159ba0 2023-10-16T08:17:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f58ec001680 con 0x7f5900159ba0 2023-10-16T08:17:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 >> v1:172.21.15.32:6789/0 conn(0x7f5900159ba0 legacy=0x7f59000cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f0003980 0x7f58f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f00047d0 0x7f58f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f58f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900071aa0 con 0x7f58f0005190 2023-10-16T08:17:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f000c030 con 0x7f58f00047d0 2023-10-16T08:17:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f000c1f0 con 0x7f58f0003980 2023-10-16T08:17:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590510f640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f58f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590490e640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f0003980 0x7f58f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58fffff640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f00047d0 0x7f58f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590510f640 1 -- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f0003980 msgr2=0x7f58f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590510f640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f58f0003980 0x7f58f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590510f640 1 -- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f00047d0 msgr2=0x7f58f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:03.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590510f640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f00047d0 0x7f58f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f590510f640 1 -- 172.21.15.32:0/1223460728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0003530 con 0x7f58f0005190 2023-10-16T08:17:03.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.414+0000 7f58fffff640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f00047d0 0x7f58f0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f590510f640 1 -- 172.21.15.32:0/1223460728 reap_dead start 2023-10-16T08:17:03.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f590510f640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f58f000b990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001e70 tx=0x7f58f801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bc0df5e1814622 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8003b80 con 0x7f58f0005190 2023-10-16T08:17:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8003d20 con 0x7f58f0005190 2023-10-16T08:17:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f58ff7fe640 1 -- 172.21.15.32:0/1223460728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8005260 con 0x7f58f0005190 2023-10-16T08:17:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 -- 172.21.15.32:0/1223460728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 msgr2=0x7f58f000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f58f000b990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001e70 tx=0x7f58f801c760 comp rx=0 tx=0).stop 2023-10-16T08:17:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 -- 172.21.15.32:0/1223460728 shutdown_connections 2023-10-16T08:17:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 --2- 172.21.15.32:0/1223460728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f58f000b990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 -- 172.21.15.32:0/1223460728 >> 172.21.15.32:0/1223460728 conn(0x7f590006c190 msgr2=0x7f590006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 -- 172.21.15.32:0/1223460728 shutdown_connections 2023-10-16T08:17:03.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 -- 172.21.15.32:0/1223460728 wait complete. 2023-10-16T08:17:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 Processor -- start 2023-10-16T08:17:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.418+0000 7f5906b72640 1 -- start start 2023-10-16T08:17:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f59001efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f590014f8d0 0x7f59000b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59000b1340 0x7f59000b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b17c0 con 0x7f58f0005190 2023-10-16T08:17:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900156810 con 0x7f59000b1340 2023-10-16T08:17:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5900156af0 con 0x7f590014f8d0 2023-10-16T08:17:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f59001efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f59001efaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39798/0 (socket says 172.21.15.32:39798) 2023-10-16T08:17:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 -- 172.21.15.32:0/4284091158 learned_addr learned my addr 172.21.15.32:0/4284091158 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f58fffff640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f590014f8d0 0x7f59000b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 -- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f590014f8d0 msgr2=0x7f59000b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f590014f8d0 0x7f59000b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 -- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59000b1340 msgr2=0x7f59000b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590510f640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59000b1340 0x7f59000b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59000b1340 0x7f59000b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 -- 172.21.15.32:0/4284091158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900156dd0 con 0x7f58f0005190 2023-10-16T08:17:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590510f640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59000b1340 0x7f59000b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:03.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f58fffff640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f590014f8d0 0x7f59000b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f590490e640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f59001efaa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58f4011960 tx=0x7f58f4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400faf0 con 0x7f58f0005190 2023-10-16T08:17:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400fc50 con 0x7f58f0005190 2023-10-16T08:17:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4014440 con 0x7f58f0005190 2023-10-16T08:17:03.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590014fd50 con 0x7f58f0005190 2023-10-16T08:17:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59000b25b0 con 0x7f58f0005190 2023-10-16T08:17:03.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5900074010 con 0x7f58f0005190 2023-10-16T08:17:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.422+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58f40145a0 con 0x7f58f0005190 2023-10-16T08:17:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.426+0000 7f58fd7fa640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f58e40664d0 0x7f58e4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.426+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f58f4084ac0 con 0x7f58f0005190 2023-10-16T08:17:03.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.426+0000 7f58fffff640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f58e40664d0 0x7f58e4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.426+0000 7f58fffff640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f58e40664d0 0x7f58e4068990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001ec0 tx=0x7f58ec000f20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.430+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f58f4051b70 con 0x7f58f0005190 2023-10-16T08:17:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.622+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5900156f60 con 0x7f58f0005190 2023-10-16T08:17:03.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f58fd7fa640 1 -- 172.21.15.32:0/4284091158 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f58f4022070 con 0x7f58f0005190 2023-10-16T08:17:03.630 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T08:17:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f58e40664d0 msgr2=0x7f58e4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f58e40664d0 0x7f58e4068990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58ec001ec0 tx=0x7f58ec000f20 comp rx=0 tx=0).stop 2023-10-16T08:17:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 msgr2=0x7f59001efaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f59001efaa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58f4011960 tx=0x7f58f4011e20 comp rx=0 tx=0).stop 2023-10-16T08:17:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 shutdown_connections 2023-10-16T08:17:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59000b1340 0x7f59000b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f58e40664d0 0x7f58e4068990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f590014f8d0 0x7f59000b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 --2- 172.21.15.32:0/4284091158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005190 0x7f59001efaa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 >> 172.21.15.32:0/4284091158 conn(0x7f590006c190 msgr2=0x7f5900153b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 shutdown_connections 2023-10-16T08:17:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:03.626+0000 7f5906b72640 1 -- 172.21.15.32:0/4284091158 wait complete. 2023-10-16T08:17:03.651 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:17:03.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:17:03.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2023-10-16T08:17:03.652 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-16T08:17:03.652 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:03.652 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:17:03.652 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:17:03.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:17:03.653 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-16T08:17:03.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.654 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2023-10-16T08:17:03.664 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:03.664 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:03.664 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:03.665 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:03.665 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:03.665 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:03.665 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:03.665 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:03.666 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2023-10-16T08:17:03.666 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:03.666 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:03.666 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2023-10-16T08:17:03.666 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:03.667 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:03.667 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:17:03.667 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:03.667 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T08:17:03.667 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42395sec preferred_lft 42395sec 2023-10-16T08:17:03.667 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::ec4:7aff:fe88:6df5/64 scope link 2023-10-16T08:17:03.668 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:03.668 INFO:teuthology.orchestra.run.smithi186.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:17:03.668 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 02:42:67:e6:90:8f brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:03.668 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:17:03.668 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:03.669 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T08:17:03.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.670 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:03.670 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:03.671 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add name ceph-brx type bridge 2023-10-16T08:17:03.671 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr flush dev ceph-brx 2023-10-16T08:17:03.671 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set ceph-brx up 2023-10-16T08:17:03.672 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T08:17:03.672 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:03.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.792 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:17:03.812 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-10-16T08:17:03.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.815 DEBUG:teuthology.orchestra.run.smithi186:> route 2023-10-16T08:17:03.893 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2023-10-16T08:17:03.893 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:17:03.893 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T08:17:03.894 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T08:17:03.894 INFO:teuthology.orchestra.run.smithi186.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:17:03.894 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T08:17:03.894 INFO:teuthology.orchestra.run.smithi186.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:17:03.894 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:17:03.895 INFO:teuthology.orchestra.run.smithi186.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:17:03.895 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:17:03.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.896 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:03.896 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:03.896 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T08:17:03.896 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T08:17:03.897 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T08:17:03.897 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:03.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.987 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-10-16T08:17:03.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:03.997 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2023-10-16T08:17:04.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:04.051 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:04.052 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:04.052 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:04.052 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-16T08:17:04.052 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:04.135 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-16T08:17:04.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:04.136 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:04.136 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:04.137 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-16T08:17:04.137 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T08:17:04.137 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-16T08:17:04.138 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-16T08:17:04.138 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-16T08:17:04.138 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:04.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:04.301 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:04.301 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:04.302 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.0 up 2023-10-16T08:17:04.302 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T08:17:04.302 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:04.343 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-16T08:17:04.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:04.344 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:04.394 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-16T08:17:04.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:04.395 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:04.445 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2023-10-16T08:17:04.511 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:04.563 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2023-10-16T08:17:04.564 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2023-10-16T08:17:04.564 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2023-10-16T08:17:04.564 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2023-10-16T08:17:04.564 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2023-10-16T08:17:04.564 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2023-10-16T08:17:04.565 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2023-10-16T08:17:04.565 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2023-10-16T08:17:04.565 INFO:teuthology.orchestra.run.smithi186.stdout:/run/lock 2023-10-16T08:17:04.565 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2023-10-16T08:17:04.565 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2023-10-16T08:17:04.565 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2023-10-16T08:17:04.566 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:04.566 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2023-10-16T08:17:04.566 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2023-10-16T08:17:04.566 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2023-10-16T08:17:04.566 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2023-10-16T08:17:04.567 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/fs/nfsd 2023-10-16T08:17:04.567 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:04.567 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2023-10-16T08:17:04.567 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:04.567 INFO:teuthology.orchestra.run.smithi186.stdout:/run/rpc_pipefs 2023-10-16T08:17:04.568 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2023-10-16T08:17:04.568 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2023-10-16T08:17:04.568 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:17:04.568 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:17:04.568 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2023-10-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:04.569 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:04.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:04.570 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2023-10-16T08:17:04.622 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:17:04.622 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-16T08:17:04.669 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2023-10-16T08:17:04.690 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:04.744 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2023-10-16T08:17:04.744 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2023-10-16T08:17:04.744 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2023-10-16T08:17:04.744 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2023-10-16T08:17:04.745 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2023-10-16T08:17:04.745 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2023-10-16T08:17:04.745 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2023-10-16T08:17:04.745 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2023-10-16T08:17:04.745 INFO:teuthology.orchestra.run.smithi186.stdout:/run/lock 2023-10-16T08:17:04.746 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2023-10-16T08:17:04.746 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2023-10-16T08:17:04.746 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2023-10-16T08:17:04.746 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:04.746 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2023-10-16T08:17:04.746 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2023-10-16T08:17:04.747 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2023-10-16T08:17:04.747 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2023-10-16T08:17:04.747 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/fs/nfsd 2023-10-16T08:17:04.747 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:04.747 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2023-10-16T08:17:04.748 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:04.748 INFO:teuthology.orchestra.run.smithi186.stdout:/run/rpc_pipefs 2023-10-16T08:17:04.748 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2023-10-16T08:17:04.748 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2023-10-16T08:17:04.748 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:17:04.748 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:17:04.749 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:17:04.749 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:17:04.749 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2023-10-16T08:17:04.749 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:04.749 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:04.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:04.751 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2023-10-16T08:17:04.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi186.stderr:2023-10-16T08:17:04.768+0000 7feafe446580 -1 init, newargv = 0x555fffa3bcf0 newargc=15 2023-10-16T08:17:04.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi186.stderr:ceph-fuse[44062]: starting ceph client 2023-10-16T08:17:04.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi186.stderr:ceph-fuse[44062]: starting fuse 2023-10-16T08:17:05.881 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2023-10-16T08:17:05.906 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:05.960 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2023-10-16T08:17:05.961 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2023-10-16T08:17:05.961 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2023-10-16T08:17:05.961 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2023-10-16T08:17:05.961 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2023-10-16T08:17:05.962 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2023-10-16T08:17:05.962 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2023-10-16T08:17:05.962 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2023-10-16T08:17:05.962 INFO:teuthology.orchestra.run.smithi186.stdout:/run/lock 2023-10-16T08:17:05.962 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2023-10-16T08:17:05.963 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2023-10-16T08:17:05.963 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2023-10-16T08:17:05.963 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:05.963 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2023-10-16T08:17:05.963 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2023-10-16T08:17:05.964 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2023-10-16T08:17:05.964 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2023-10-16T08:17:05.964 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/fs/nfsd 2023-10-16T08:17:05.964 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:05.964 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2023-10-16T08:17:05.965 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:05.965 INFO:teuthology.orchestra.run.smithi186.stdout:/run/rpc_pipefs 2023-10-16T08:17:05.965 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2023-10-16T08:17:05.965 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2023-10-16T08:17:05.965 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:17:05.966 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:17:05.966 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:17:05.966 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:17:05.966 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2023-10-16T08:17:05.966 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:05.967 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:05.967 INFO:teuthology.orchestra.run.smithi186.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:05.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:05.968 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2023-10-16T08:17:06.019 INFO:teuthology.orchestra.run.smithi186.stdout:43 2023-10-16T08:17:06.019 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:17:06.019 DEBUG:teuthology.orchestra.run.smithi186:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:17:06.020 DEBUG:teuthology.orchestra.run.smithi186:> import glob 2023-10-16T08:17:06.020 DEBUG:teuthology.orchestra.run.smithi186:> import re 2023-10-16T08:17:06.020 DEBUG:teuthology.orchestra.run.smithi186:> import os 2023-10-16T08:17:06.021 DEBUG:teuthology.orchestra.run.smithi186:> import subprocess 2023-10-16T08:17:06.021 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:06.021 DEBUG:teuthology.orchestra.run.smithi186:> def _find_admin_socket(client_name): 2023-10-16T08:17:06.021 DEBUG:teuthology.orchestra.run.smithi186:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-16T08:17:06.022 DEBUG:teuthology.orchestra.run.smithi186:> files = glob.glob(asok_path) 2023-10-16T08:17:06.022 DEBUG:teuthology.orchestra.run.smithi186:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-16T08:17:06.022 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:06.023 DEBUG:teuthology.orchestra.run.smithi186:> # Given a non-glob path, it better be there 2023-10-16T08:17:06.023 DEBUG:teuthology.orchestra.run.smithi186:> if "*" not in asok_path: 2023-10-16T08:17:06.023 DEBUG:teuthology.orchestra.run.smithi186:> assert(len(files) == 1) 2023-10-16T08:17:06.023 DEBUG:teuthology.orchestra.run.smithi186:> return files[0] 2023-10-16T08:17:06.024 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:06.024 DEBUG:teuthology.orchestra.run.smithi186:> for f in files: 2023-10-16T08:17:06.024 DEBUG:teuthology.orchestra.run.smithi186:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:17:06.025 DEBUG:teuthology.orchestra.run.smithi186:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:17:06.025 DEBUG:teuthology.orchestra.run.smithi186:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:17:06.025 DEBUG:teuthology.orchestra.run.smithi186:> contents = proc_f.read() 2023-10-16T08:17:06.026 DEBUG:teuthology.orchestra.run.smithi186:> if mountpoint in contents: 2023-10-16T08:17:06.026 DEBUG:teuthology.orchestra.run.smithi186:> return f 2023-10-16T08:17:06.026 DEBUG:teuthology.orchestra.run.smithi186:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:17:06.026 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:06.027 DEBUG:teuthology.orchestra.run.smithi186:> print(_find_admin_socket("client.1")) 2023-10-16T08:17:06.027 DEBUG:teuthology.orchestra.run.smithi186:> ' 2023-10-16T08:17:06.155 INFO:teuthology.orchestra.run.smithi186.stdout:/var/run/ceph/ceph-client.1.44062.asok 2023-10-16T08:17:06.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:17:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:17:06.167 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44062.asok 2023-10-16T08:17:06.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.168 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44062.asok status 2023-10-16T08:17:06.340 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T08:17:06.340 INFO:teuthology.orchestra.run.smithi186.stdout: "metadata": { 2023-10-16T08:17:06.340 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:17:06.341 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:17:06.341 INFO:teuthology.orchestra.run.smithi186.stdout: "entity_id": "1", 2023-10-16T08:17:06.341 INFO:teuthology.orchestra.run.smithi186.stdout: "hostname": "smithi186", 2023-10-16T08:17:06.341 INFO:teuthology.orchestra.run.smithi186.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-16T08:17:06.341 INFO:teuthology.orchestra.run.smithi186.stdout: "pid": "44062", 2023-10-16T08:17:06.342 INFO:teuthology.orchestra.run.smithi186.stdout: "root": "/" 2023-10-16T08:17:06.342 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:06.342 INFO:teuthology.orchestra.run.smithi186.stdout: "dentry_count": 0, 2023-10-16T08:17:06.342 INFO:teuthology.orchestra.run.smithi186.stdout: "dentry_pinned_count": 0, 2023-10-16T08:17:06.342 INFO:teuthology.orchestra.run.smithi186.stdout: "id": 4849, 2023-10-16T08:17:06.342 INFO:teuthology.orchestra.run.smithi186.stdout: "inst": { 2023-10-16T08:17:06.343 INFO:teuthology.orchestra.run.smithi186.stdout: "name": { 2023-10-16T08:17:06.343 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "client", 2023-10-16T08:17:06.343 INFO:teuthology.orchestra.run.smithi186.stdout: "num": 4849 2023-10-16T08:17:06.343 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:06.343 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": { 2023-10-16T08:17:06.343 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "v1", 2023-10-16T08:17:06.344 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": "172.21.15.186:0", 2023-10-16T08:17:06.344 INFO:teuthology.orchestra.run.smithi186.stdout: "nonce": 1880880624 2023-10-16T08:17:06.344 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T08:17:06.344 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:06.344 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": { 2023-10-16T08:17:06.344 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "v1", 2023-10-16T08:17:06.345 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": "172.21.15.186:0", 2023-10-16T08:17:06.345 INFO:teuthology.orchestra.run.smithi186.stdout: "nonce": 1880880624 2023-10-16T08:17:06.345 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:06.345 INFO:teuthology.orchestra.run.smithi186.stdout: "inst_str": "client.4849 172.21.15.186:0/1880880624", 2023-10-16T08:17:06.345 INFO:teuthology.orchestra.run.smithi186.stdout: "addr_str": "172.21.15.186:0/1880880624", 2023-10-16T08:17:06.345 INFO:teuthology.orchestra.run.smithi186.stdout: "inode_count": 1, 2023-10-16T08:17:06.346 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_epoch": 9, 2023-10-16T08:17:06.346 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_epoch": 27, 2023-10-16T08:17:06.346 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:17:06.346 INFO:teuthology.orchestra.run.smithi186.stdout: "blocklisted": false, 2023-10-16T08:17:06.346 INFO:teuthology.orchestra.run.smithi186.stdout: "fs_name": "cephfs" 2023-10-16T08:17:06.347 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T08:17:06.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:17:06.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.502+0000 7f9e69dd9640 1 Processor -- start 2023-10-16T08:17:06.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.502+0000 7f9e69dd9640 1 -- start start 2023-10-16T08:17:06.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e69dd9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e641597c0 0x7f9e640ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:06.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e69dd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e64159ba0 0x7f9e640cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:06.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e69dd9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640723d0 con 0x7f9e640adc10 2023-10-16T08:17:06.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e69dd9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e640717c0 con 0x7f9e641597c0 2023-10-16T08:17:06.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e69dd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64071aa0 con 0x7f9e64159ba0 2023-10-16T08:17:06.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e641597c0 0x7f9e640ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:06.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e641597c0 0x7f9e640ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39808/0 (socket says 172.21.15.32:39808) 2023-10-16T08:17:06.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 -- 172.21.15.32:0/2637923669 learned_addr learned my addr 172.21.15.32:0/2637923669 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:06.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e62ffd640 1 --2- 172.21.15.32:0/2637923669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e64159ba0 0x7f9e640cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:06.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 -- 172.21.15.32:0/2637923669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e64159ba0 msgr2=0x7f9e640cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 --2- 172.21.15.32:0/2637923669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e64159ba0 0x7f9e640cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 -- 172.21.15.32:0/2637923669 >> v1:172.21.15.186:6789/0 conn(0x7f9e640adc10 legacy=0x7f9e640cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.506+0000 7f9e637fe640 1 -- 172.21.15.32:0/2637923669 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64071d80 con 0x7f9e641597c0 2023-10-16T08:17:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e637fe640 1 --2- 172.21.15.32:0/2637923669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e641597c0 0x7f9e640ad6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009870 tx=0x7f9e5401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a442f157bceffe7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:06.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e627fc640 1 -- 172.21.15.32:0/2637923669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540aa640 con 0x7f9e641597c0 2023-10-16T08:17:06.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e627fc640 1 -- 172.21.15.32:0/2637923669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e540aa7a0 con 0x7f9e641597c0 2023-10-16T08:17:06.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e627fc640 1 -- 172.21.15.32:0/2637923669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540aab30 con 0x7f9e641597c0 2023-10-16T08:17:06.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 -- 172.21.15.32:0/2637923669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e641597c0 msgr2=0x7f9e640ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:06.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/2637923669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e641597c0 0x7f9e640ad6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009870 tx=0x7f9e5401c540 comp rx=0 tx=0).stop 2023-10-16T08:17:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 -- 172.21.15.32:0/2637923669 shutdown_connections 2023-10-16T08:17:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/2637923669 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e64159ba0 0x7f9e640cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/2637923669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e641597c0 0x7f9e640ad6d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 -- 172.21.15.32:0/2637923669 >> 172.21.15.32:0/2637923669 conn(0x7f9e6406c190 msgr2=0x7f9e6406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 -- 172.21.15.32:0/2637923669 shutdown_connections 2023-10-16T08:17:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 -- 172.21.15.32:0/2637923669 wait complete. 2023-10-16T08:17:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 Processor -- start 2023-10-16T08:17:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.510+0000 7f9e69dd9640 1 -- start start 2023-10-16T08:17:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 0x7f9e64149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e641597c0 0x7f9e640ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e64159ba0 0x7f9e640ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64159fc0 con 0x7f9e640adc10 2023-10-16T08:17:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6415ae20 con 0x7f9e64159ba0 2023-10-16T08:17:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6414a070 con 0x7f9e641597c0 2023-10-16T08:17:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 0x7f9e64149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 0x7f9e64149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39818/0 (socket says 172.21.15.32:39818) 2023-10-16T08:17:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 -- 172.21.15.32:0/1386834067 learned_addr learned my addr 172.21.15.32:0/1386834067 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e62ffd640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e641597c0 0x7f9e640ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e63fff640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e64159ba0 0x7f9e640ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 -- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e641597c0 msgr2=0x7f9e640ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e641597c0 0x7f9e640ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 -- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e64159ba0 msgr2=0x7f9e640ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e64159ba0 0x7f9e640ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 -- 172.21.15.32:0/1386834067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540094d0 con 0x7f9e640adc10 2023-10-16T08:17:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e637fe640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 0x7f9e64149b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e540ac040 tx=0x7f9e5401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540bd5f0 con 0x7f9e640adc10 2023-10-16T08:17:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e540bd770 con 0x7f9e640adc10 2023-10-16T08:17:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e640af2f0 con 0x7f9e640adc10 2023-10-16T08:17:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540bdb20 con 0x7f9e640adc10 2023-10-16T08:17:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6415a4f0 con 0x7f9e640adc10 2023-10-16T08:17:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.514+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6415af90 con 0x7f9e640adc10 2023-10-16T08:17:06.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.518+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e540a8050 con 0x7f9e640adc10 2023-10-16T08:17:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.518+0000 7f9e4bfff640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e500664a0 0x7f9e50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.518+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e5411ebe0 con 0x7f9e640adc10 2023-10-16T08:17:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.518+0000 7f9e62ffd640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e500664a0 0x7f9e50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.518+0000 7f9e62ffd640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e500664a0 0x7f9e50068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e640ae1e0 tx=0x7f9e58009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:06.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.522+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e540ebdd0 con 0x7f9e640adc10 2023-10-16T08:17:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9e64074010 con 0x7f9e640adc10 2023-10-16T08:17:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e4bfff640 1 -- 172.21.15.32:0/1386834067 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9e540ef3f0 con 0x7f9e640adc10 2023-10-16T08:17:06.716 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T08:17:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e500664a0 msgr2=0x7f9e50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e500664a0 0x7f9e50068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e640ae1e0 tx=0x7f9e58009190 comp rx=0 tx=0).stop 2023-10-16T08:17:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 msgr2=0x7f9e64149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 0x7f9e64149b30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e540ac040 tx=0x7f9e5401ad80 comp rx=0 tx=0).stop 2023-10-16T08:17:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 shutdown_connections 2023-10-16T08:17:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e64159ba0 0x7f9e640ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e500664a0 0x7f9e50068960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e641597c0 0x7f9e640ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 --2- 172.21.15.32:0/1386834067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e640adc10 0x7f9e64149b30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.710+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 >> 172.21.15.32:0/1386834067 conn(0x7f9e6406c190 msgr2=0x7f9e640cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.714+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 shutdown_connections 2023-10-16T08:17:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:06.714+0000 7f9e69dd9640 1 -- 172.21.15.32:0/1386834067 wait complete. 2023-10-16T08:17:06.737 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:17:06.738 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:17:06.738 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2023-10-16T08:17:06.738 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-16T08:17:06.738 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:06.739 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:17:06.739 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:17:06.739 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:17:06.739 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-16T08:17:06.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.740 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2023-10-16T08:17:06.750 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:06.750 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:06.750 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:06.750 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.751 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:06.751 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.751 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:06.751 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:06.751 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2023-10-16T08:17:06.751 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:06.752 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:06.752 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2023-10-16T08:17:06.752 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:06.752 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:06.752 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:17:06.753 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:06.753 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T08:17:06.753 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 42391sec preferred_lft 42391sec 2023-10-16T08:17:06.753 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::ec4:7aff:fe88:6df5/64 scope link 2023-10-16T08:17:06.753 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.754 INFO:teuthology.orchestra.run.smithi186.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:17:06.754 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 02:42:67:e6:90:8f brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:06.754 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:17:06.754 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.754 INFO:teuthology.orchestra.run.smithi186.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:06.755 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:06.755 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:17:06.755 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.755 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:17:06.755 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.756 INFO:teuthology.orchestra.run.smithi186.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:17:06.756 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:06.756 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:17:06.756 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:06.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.758 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:17:06.820 INFO:teuthology.orchestra.run.smithi186.stdout:1 2023-10-16T08:17:06.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.823 DEBUG:teuthology.orchestra.run.smithi186:> route 2023-10-16T08:17:06.876 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2023-10-16T08:17:06.877 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:17:06.877 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T08:17:06.877 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T08:17:06.877 INFO:teuthology.orchestra.run.smithi186.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:17:06.878 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T08:17:06.878 INFO:teuthology.orchestra.run.smithi186.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:17:06.878 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:17:06.878 INFO:teuthology.orchestra.run.smithi186.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:17:06.879 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:17:06.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.879 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:06.879 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:06.880 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T08:17:06.880 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T08:17:06.880 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T08:17:06.881 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:06.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.971 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-10-16T08:17:06.980 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T08:17:06.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:06.981 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2023-10-16T08:17:07.030 INFO:teuthology.orchestra.run.smithi186.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-16T08:17:07.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.032 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:07.032 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:07.032 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:17:07.032 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-16T08:17:07.033 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:07.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.112 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T08:17:07.174 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:07.174 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:07.175 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:07.175 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.175 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:07.175 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.175 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:07.176 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 3e:54:60:cb:0e:2f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:17:07.176 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:17:07.176 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.176 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::3c54:60ff:fecb:e2f/64 scope link 2023-10-16T08:17:07.176 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.178 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T08:17:07.242 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:07.243 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:07.243 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:07.243 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.243 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:07.244 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.244 INFO:teuthology.orchestra.run.smithi186.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:07.244 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 3e:54:60:cb:0e:2f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:17:07.244 INFO:teuthology.orchestra.run.smithi186.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:17:07.245 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.245 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::3c54:60ff:fecb:e2f/64 scope link 2023-10-16T08:17:07.245 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:07.246 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-16T08:17:07.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.247 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:07.247 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:07.247 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-16T08:17:07.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T08:17:07.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-16T08:17:07.248 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-16T08:17:07.249 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-16T08:17:07.249 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:07.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.387 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec bash -c ' 2023-10-16T08:17:07.387 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2023-10-16T08:17:07.387 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.1 up 2023-10-16T08:17:07.388 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T08:17:07.388 DEBUG:teuthology.orchestra.run.smithi186:> ') 2023-10-16T08:17:07.427 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-16T08:17:07.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:07.428 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:07.478 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-16T08:17:07.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:07.479 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:07.529 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2023-10-16T08:17:07.591 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:07.643 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2023-10-16T08:17:07.644 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2023-10-16T08:17:07.644 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2023-10-16T08:17:07.644 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2023-10-16T08:17:07.644 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2023-10-16T08:17:07.645 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2023-10-16T08:17:07.645 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2023-10-16T08:17:07.645 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2023-10-16T08:17:07.645 INFO:teuthology.orchestra.run.smithi186.stdout:/run/lock 2023-10-16T08:17:07.645 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2023-10-16T08:17:07.646 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2023-10-16T08:17:07.646 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2023-10-16T08:17:07.646 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:07.646 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2023-10-16T08:17:07.646 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2023-10-16T08:17:07.647 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2023-10-16T08:17:07.647 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2023-10-16T08:17:07.647 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/fs/nfsd 2023-10-16T08:17:07.647 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:07.647 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2023-10-16T08:17:07.648 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:07.648 INFO:teuthology.orchestra.run.smithi186.stdout:/run/rpc_pipefs 2023-10-16T08:17:07.648 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2023-10-16T08:17:07.648 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2023-10-16T08:17:07.648 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:17:07.648 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:17:07.649 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:17:07.649 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:17:07.649 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2023-10-16T08:17:07.649 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:07.649 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:07.649 INFO:teuthology.orchestra.run.smithi186.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:07.650 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:17:07.650 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:17:07.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.651 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2023-10-16T08:17:07.701 INFO:teuthology.orchestra.run.smithi186.stdout:43 2023-10-16T08:17:07.702 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-10-16T08:17:07.702 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-16T08:17:07.749 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2023-10-16T08:17:07.770 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:07.824 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2023-10-16T08:17:07.824 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2023-10-16T08:17:07.824 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2023-10-16T08:17:07.824 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2023-10-16T08:17:07.825 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2023-10-16T08:17:07.825 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2023-10-16T08:17:07.825 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2023-10-16T08:17:07.825 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2023-10-16T08:17:07.825 INFO:teuthology.orchestra.run.smithi186.stdout:/run/lock 2023-10-16T08:17:07.826 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2023-10-16T08:17:07.826 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2023-10-16T08:17:07.826 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2023-10-16T08:17:07.826 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:07.826 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2023-10-16T08:17:07.827 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2023-10-16T08:17:07.827 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2023-10-16T08:17:07.827 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2023-10-16T08:17:07.827 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/fs/nfsd 2023-10-16T08:17:07.827 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:07.828 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2023-10-16T08:17:07.828 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:07.828 INFO:teuthology.orchestra.run.smithi186.stdout:/run/rpc_pipefs 2023-10-16T08:17:07.828 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2023-10-16T08:17:07.828 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2023-10-16T08:17:07.828 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:17:07.829 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:17:07.829 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:17:07.829 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:17:07.829 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2023-10-16T08:17:07.829 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:07.829 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:17:07.830 INFO:teuthology.orchestra.run.smithi186.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:07.830 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:17:07.830 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:17:07.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.831 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2023-10-16T08:17:07.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi186.stderr:2023-10-16T08:17:07.839+0000 7f955263a580 -1 init, newargv = 0x55e2d3b54d20 newargc=15 2023-10-16T08:17:07.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi186.stderr:ceph-fuse[44176]: starting ceph client 2023-10-16T08:17:07.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi186.stderr:ceph-fuse[44176]: starting fuse 2023-10-16T08:17:07.853 INFO:teuthology.orchestra.run.smithi186.stdout:43 2023-10-16T08:17:07.853 INFO:teuthology.orchestra.run.smithi186.stdout:44 2023-10-16T08:17:07.853 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-10-16T08:17:07.854 DEBUG:teuthology.orchestra.run.smithi186:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:17:07.854 DEBUG:teuthology.orchestra.run.smithi186:> import glob 2023-10-16T08:17:07.855 DEBUG:teuthology.orchestra.run.smithi186:> import re 2023-10-16T08:17:07.855 DEBUG:teuthology.orchestra.run.smithi186:> import os 2023-10-16T08:17:07.855 DEBUG:teuthology.orchestra.run.smithi186:> import subprocess 2023-10-16T08:17:07.855 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:07.856 DEBUG:teuthology.orchestra.run.smithi186:> def _find_admin_socket(client_name): 2023-10-16T08:17:07.856 DEBUG:teuthology.orchestra.run.smithi186:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-16T08:17:07.856 DEBUG:teuthology.orchestra.run.smithi186:> files = glob.glob(asok_path) 2023-10-16T08:17:07.857 DEBUG:teuthology.orchestra.run.smithi186:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-16T08:17:07.857 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:07.857 DEBUG:teuthology.orchestra.run.smithi186:> # Given a non-glob path, it better be there 2023-10-16T08:17:07.857 DEBUG:teuthology.orchestra.run.smithi186:> if "*" not in asok_path: 2023-10-16T08:17:07.858 DEBUG:teuthology.orchestra.run.smithi186:> assert(len(files) == 1) 2023-10-16T08:17:07.858 DEBUG:teuthology.orchestra.run.smithi186:> return files[0] 2023-10-16T08:17:07.858 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:07.859 DEBUG:teuthology.orchestra.run.smithi186:> for f in files: 2023-10-16T08:17:07.859 DEBUG:teuthology.orchestra.run.smithi186:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:17:07.859 DEBUG:teuthology.orchestra.run.smithi186:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:17:07.860 DEBUG:teuthology.orchestra.run.smithi186:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:17:07.860 DEBUG:teuthology.orchestra.run.smithi186:> contents = proc_f.read() 2023-10-16T08:17:07.860 DEBUG:teuthology.orchestra.run.smithi186:> if mountpoint in contents: 2023-10-16T08:17:07.861 DEBUG:teuthology.orchestra.run.smithi186:> return f 2023-10-16T08:17:07.861 DEBUG:teuthology.orchestra.run.smithi186:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:17:07.861 DEBUG:teuthology.orchestra.run.smithi186:> 2023-10-16T08:17:07.861 DEBUG:teuthology.orchestra.run.smithi186:> print(_find_admin_socket("client.3")) 2023-10-16T08:17:07.862 DEBUG:teuthology.orchestra.run.smithi186:> ' 2023-10-16T08:17:07.977 INFO:teuthology.orchestra.run.smithi186.stdout:/var/run/ceph/ceph-client.3.44176.asok 2023-10-16T08:17:07.981 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T08:17:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:17:07.989 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44176.asok 2023-10-16T08:17:07.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:07.989 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44176.asok status 2023-10-16T08:17:08.162 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T08:17:08.162 INFO:teuthology.orchestra.run.smithi186.stdout: "metadata": { 2023-10-16T08:17:08.162 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:17:08.162 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:17:08.163 INFO:teuthology.orchestra.run.smithi186.stdout: "entity_id": "3", 2023-10-16T08:17:08.163 INFO:teuthology.orchestra.run.smithi186.stdout: "hostname": "smithi186", 2023-10-16T08:17:08.163 INFO:teuthology.orchestra.run.smithi186.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-16T08:17:08.163 INFO:teuthology.orchestra.run.smithi186.stdout: "pid": "44176", 2023-10-16T08:17:08.163 INFO:teuthology.orchestra.run.smithi186.stdout: "root": "/" 2023-10-16T08:17:08.164 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:08.164 INFO:teuthology.orchestra.run.smithi186.stdout: "dentry_count": 0, 2023-10-16T08:17:08.164 INFO:teuthology.orchestra.run.smithi186.stdout: "dentry_pinned_count": 0, 2023-10-16T08:17:08.164 INFO:teuthology.orchestra.run.smithi186.stdout: "id": 4643, 2023-10-16T08:17:08.164 INFO:teuthology.orchestra.run.smithi186.stdout: "inst": { 2023-10-16T08:17:08.165 INFO:teuthology.orchestra.run.smithi186.stdout: "name": { 2023-10-16T08:17:08.165 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "client", 2023-10-16T08:17:08.165 INFO:teuthology.orchestra.run.smithi186.stdout: "num": 4643 2023-10-16T08:17:08.165 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:08.165 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": { 2023-10-16T08:17:08.165 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "v1", 2023-10-16T08:17:08.166 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:17:08.166 INFO:teuthology.orchestra.run.smithi186.stdout: "nonce": 3661493870 2023-10-16T08:17:08.166 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T08:17:08.166 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:08.166 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": { 2023-10-16T08:17:08.167 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "v1", 2023-10-16T08:17:08.167 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:17:08.167 INFO:teuthology.orchestra.run.smithi186.stdout: "nonce": 3661493870 2023-10-16T08:17:08.167 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T08:17:08.167 INFO:teuthology.orchestra.run.smithi186.stdout: "inst_str": "client.4643 192.168.0.2:0/3661493870", 2023-10-16T08:17:08.167 INFO:teuthology.orchestra.run.smithi186.stdout: "addr_str": "192.168.0.2:0/3661493870", 2023-10-16T08:17:08.168 INFO:teuthology.orchestra.run.smithi186.stdout: "inode_count": 1, 2023-10-16T08:17:08.168 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_epoch": 9, 2023-10-16T08:17:08.168 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_epoch": 27, 2023-10-16T08:17:08.168 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:17:08.168 INFO:teuthology.orchestra.run.smithi186.stdout: "blocklisted": false, 2023-10-16T08:17:08.169 INFO:teuthology.orchestra.run.smithi186.stdout: "fs_name": "cephfs" 2023-10-16T08:17:08.169 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T08:17:08.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.176 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:17:08.177 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:08.188 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:17:08.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:08.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:08.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.252 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:17:08.253 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:17:08.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:08.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:08.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.369 DEBUG:teuthology.orchestra.run.smithi186:> stat --file-system '--printf=%T 2023-10-16T08:17:08.369 DEBUG:teuthology.orchestra.run.smithi186:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:08.378 INFO:teuthology.orchestra.run.smithi186.stdout:fuseblk 2023-10-16T08:17:08.379 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:08.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.379 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:08.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.442 DEBUG:teuthology.orchestra.run.smithi186:> stat --file-system '--printf=%T 2023-10-16T08:17:08.443 DEBUG:teuthology.orchestra.run.smithi186:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:08.494 INFO:teuthology.orchestra.run.smithi186.stdout:fuseblk 2023-10-16T08:17:08.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:08.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:08.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:08.558 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-16T08:17:08.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:08.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10c93e640 1 Processor -- start 2023-10-16T08:17:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10c93e640 1 -- start start 2023-10-16T08:17:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10c93e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd10815ca70 0x7fd1080d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10c93e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080c65f0 con 0x7fd1080d3550 2023-10-16T08:17:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10c93e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080cdf80 con 0x7fd1080cf7a0 2023-10-16T08:17:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10c93e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1080ce820 con 0x7fd10815ca70 2023-10-16T08:17:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd105d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1080d3550 0x7fd1080cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37222/0 (socket says 172.21.15.32:37222) 2023-10-16T08:17:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd105d9b640 1 -- 172.21.15.32:0/2874411735 learned_addr learned my addr 172.21.15.32:0/2874411735 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10659c640 1 --2- 172.21.15.32:0/2874411735 >> v2:172.21.15.186:3300/0 conn(0x7fd10815ca70 0x7fd1080d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912367477 0 0) 0x7fd1080c65f0 con 0x7fd1080d3550 2023-10-16T08:17:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f0003530 con 0x7fd1080d3550 2023-10-16T08:17:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818534697 0 0) 0x7fd1080cdf80 con 0x7fd1080cf7a0 2023-10-16T08:17:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1080c65f0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10659c640 1 -- 172.21.15.32:0/2874411735 >> v1:172.21.15.186:6789/0 conn(0x7fd1080cf7a0 legacy=0x7fd1080c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10659c640 1 -- 172.21.15.32:0/2874411735 >> v1:172.21.15.32:6789/0 conn(0x7fd1080d3550 legacy=0x7fd1080cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051620546 0 0) 0x7fd0f0003530 con 0x7fd1080d3550 2023-10-16T08:17:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.722+0000 7fd10659c640 1 -- 172.21.15.32:0/2874411735 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080ce260 con 0x7fd10815ca70 2023-10-16T08:17:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10659c640 1 --2- 172.21.15.32:0/2874411735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd1080d3010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097a0 tx=0x7fd0f801c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5464c2e7f4e1801 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc001680 con 0x7fd10815ca70 2023-10-16T08:17:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80a7050 con 0x7fd10815ca70 2023-10-16T08:17:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10559a640 1 -- 172.21.15.32:0/2874411735 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80ab620 con 0x7fd10815ca70 2023-10-16T08:17:08.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 -- 172.21.15.32:0/2874411735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 msgr2=0x7fd1080d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 --2- 172.21.15.32:0/2874411735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd1080d3010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097a0 tx=0x7fd0f801c350 comp rx=0 tx=0).stop 2023-10-16T08:17:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 -- 172.21.15.32:0/2874411735 shutdown_connections 2023-10-16T08:17:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 --2- 172.21.15.32:0/2874411735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd1080d3010 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 -- 172.21.15.32:0/2874411735 >> 172.21.15.32:0/2874411735 conn(0x7fd10809bc50 msgr2=0x7fd1080d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:08.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 -- 172.21.15.32:0/2874411735 shutdown_connections 2023-10-16T08:17:08.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.726+0000 7fd10c93e640 1 -- 172.21.15.32:0/2874411735 wait complete. 2023-10-16T08:17:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 Processor -- start 2023-10-16T08:17:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 -- start start 2023-10-16T08:17:08.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 0x7fd108156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1080d3550 0x7fd108153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd10814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108156610 con 0x7fd1080cf7a0 2023-10-16T08:17:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108156330 con 0x7fd10815ca70 2023-10-16T08:17:08.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10c93e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd108150e70 con 0x7fd1080d3550 2023-10-16T08:17:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 0x7fd108156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 0x7fd108156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39832/0 (socket says 172.21.15.32:39832) 2023-10-16T08:17:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 -- 172.21.15.32:0/265081475 learned_addr learned my addr 172.21.15.32:0/265081475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd105d9b640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1080d3550 0x7fd108153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd106d9d640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd10814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:08.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 -- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1080d3550 msgr2=0x7fd108153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1080d3550 0x7fd108153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 -- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 msgr2=0x7fd10814c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd10814c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 -- 172.21.15.32:0/265081475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f80093b0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd105d9b640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1080d3550 0x7fd108153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.730+0000 7fd10659c640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 0x7fd108156d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097a0 tx=0x7fd0f80190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80059a0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8005b00 con 0x7fd1080cf7a0 2023-10-16T08:17:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004030 con 0x7fd1080cf7a0 2023-10-16T08:17:08.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd108151150 con 0x7fd1080cf7a0 2023-10-16T08:17:08.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd108157250 con 0x7fd1080cf7a0 2023-10-16T08:17:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108150cd0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0f80b2040 con 0x7fd1080cf7a0 2023-10-16T08:17:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd0f6ffd640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0e80664a0 0x7fd0e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd0f811d500 con 0x7fd1080cf7a0 2023-10-16T08:17:08.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd105d9b640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0e80664a0 0x7fd0e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.734+0000 7fd105d9b640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0e80664a0 0x7fd0e8068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd108156770 tx=0x7fd0fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:08.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.738+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd108150cd0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.898+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd108150cd0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd0f6ffd640 1 -- 172.21.15.32:0/265081475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fd108150cd0 con 0x7fd1080cf7a0 2023-10-16T08:17:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:08.906 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0e80664a0 msgr2=0x7fd0e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0e80664a0 0x7fd0e8068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd108156770 tx=0x7fd0fc009040 comp rx=0 tx=0).stop 2023-10-16T08:17:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 msgr2=0x7fd108156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 0x7fd108156d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80097a0 tx=0x7fd0f80190f0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 shutdown_connections 2023-10-16T08:17:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10815ca70 0x7fd10814c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0e80664a0 0x7fd0e8068960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1080d3550 0x7fd108153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 --2- 172.21.15.32:0/265081475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1080cf7a0 0x7fd108156d10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 >> 172.21.15.32:0/265081475 conn(0x7fd10809bc50 msgr2=0x7fd10806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 shutdown_connections 2023-10-16T08:17:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:08.902+0000 7fd10c93e640 1 -- 172.21.15.32:0/265081475 wait complete. 2023-10-16T08:17:08.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:08.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:09.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 Processor -- start 2023-10-16T08:17:09.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 -- start start 2023-10-16T08:17:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7840d0210 0x7fb7840d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7840b84e0 0x7fb7840b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7840d1670 con 0x7fb78415cba0 2023-10-16T08:17:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7840d1460 con 0x7fb7840b84e0 2023-10-16T08:17:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840d1950 con 0x7fb7840d0210 2023-10-16T08:17:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb783fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb78415cba0 0x7fb7840cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37238/0 (socket says 172.21.15.32:37238) 2023-10-16T08:17:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb783fff640 1 -- 172.21.15.32:0/2060613752 learned_addr learned my addr 172.21.15.32:0/2060613752 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb788e66640 1 --2- 172.21.15.32:0/2060613752 >> v2:172.21.15.32:3300/0 conn(0x7fb7840b84e0 0x7fb7840b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb7837fe640 1 --2- 172.21.15.32:0/2060613752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7840d0210 0x7fb7840d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb788e66640 1 -- 172.21.15.32:0/2060613752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7840d0210 msgr2=0x7fb7840d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb788e66640 1 --2- 172.21.15.32:0/2060613752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7840d0210 0x7fb7840d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb788e66640 1 -- 172.21.15.32:0/2060613752 >> v1:172.21.15.32:6789/0 conn(0x7fb78415cba0 legacy=0x7fb7840cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb788e66640 1 -- 172.21.15.32:0/2060613752 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840d1c30 con 0x7fb7840b84e0 2023-10-16T08:17:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.070+0000 7fb782ffd640 1 -- 172.21.15.32:0/2060613752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532544624 0 0) 0x7fb7840d1670 con 0x7fb78415cba0 2023-10-16T08:17:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb788e66640 1 --2- 172.21.15.32:0/2060613752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7840b5cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009d90 tx=0x7fb77c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff3a0d3a7bbcb5e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb782ffd640 1 -- 172.21.15.32:0/2060613752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0a5a10 con 0x7fb7840b84e0 2023-10-16T08:17:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb782ffd640 1 -- 172.21.15.32:0/2060613752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c0a5b70 con 0x7fb7840b84e0 2023-10-16T08:17:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb782ffd640 1 -- 172.21.15.32:0/2060613752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0ad440 con 0x7fb7840b84e0 2023-10-16T08:17:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 -- 172.21.15.32:0/2060613752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 msgr2=0x7fb7840b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/2060613752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7840b5cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009d90 tx=0x7fb77c01c950 comp rx=0 tx=0).stop 2023-10-16T08:17:09.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 -- 172.21.15.32:0/2060613752 shutdown_connections 2023-10-16T08:17:09.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/2060613752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7840b5cf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/2060613752 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb7840d0210 0x7fb7840d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 -- 172.21.15.32:0/2060613752 >> 172.21.15.32:0/2060613752 conn(0x7fb7840719c0 msgr2=0x7fb784071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 -- 172.21.15.32:0/2060613752 shutdown_connections 2023-10-16T08:17:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.074+0000 7fb78a8c9640 1 -- 172.21.15.32:0/2060613752 wait complete. 2023-10-16T08:17:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 Processor -- start 2023-10-16T08:17:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 -- start start 2023-10-16T08:17:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7841572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7840d0210 0x7fb784149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb78415cba0 0x7fb784147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841482b0 con 0x7fb7840b84e0 2023-10-16T08:17:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78414b650 con 0x7fb7840d0210 2023-10-16T08:17:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb784157920 con 0x7fb78415cba0 2023-10-16T08:17:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7841572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7841572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39852/0 (socket says 172.21.15.32:39852) 2023-10-16T08:17:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 -- 172.21.15.32:0/1219391618 learned_addr learned my addr 172.21.15.32:0/1219391618 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:09.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb788e66640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb78415cba0 0x7fb784147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 -- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb78415cba0 msgr2=0x7fb784147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb78415cba0 0x7fb784147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb7837fe640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7840d0210 0x7fb784149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 -- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7840d0210 msgr2=0x7fb784149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7840d0210 0x7fb784149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 -- 172.21.15.32:0/1219391618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c009a70 con 0x7fb7840b84e0 2023-10-16T08:17:09.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb788e66640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb78415cba0 0x7fb784147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:09.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb7837fe640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7840d0210 0x7fb784149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:09.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb783fff640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7841572d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb77400c480 tx=0x7fb77400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740074e0 con 0x7fb7840b84e0 2023-10-16T08:17:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774007660 con 0x7fb7840b84e0 2023-10-16T08:17:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774004a30 con 0x7fb7840b84e0 2023-10-16T08:17:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb784148780 con 0x7fb7840b84e0 2023-10-16T08:17:09.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.078+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb784148d40 con 0x7fb7840b84e0 2023-10-16T08:17:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.082+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78404e170 con 0x7fb7840b84e0 2023-10-16T08:17:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.082+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb774004b90 con 0x7fb7840b84e0 2023-10-16T08:17:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.082+0000 7fb780ff9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb7700664a0 0x7fb770068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.082+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb774083400 con 0x7fb7840b84e0 2023-10-16T08:17:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.082+0000 7fb7837fe640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb7700664a0 0x7fb770068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.082+0000 7fb7837fe640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb7700664a0 0x7fb770068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b5e30 tx=0x7fb778008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.086+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb774053f40 con 0x7fb7840b84e0 2023-10-16T08:17:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.242+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb784154d10 con 0x7fb7840b84e0 2023-10-16T08:17:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb780ff9640 1 -- 172.21.15.32:0/1219391618 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb774053ad0 con 0x7fb7840b84e0 2023-10-16T08:17:09.251 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb7700664a0 msgr2=0x7fb770068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb7700664a0 0x7fb770068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b5e30 tx=0x7fb778008040 comp rx=0 tx=0).stop 2023-10-16T08:17:09.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 msgr2=0x7fb7841572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7841572d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb77400c480 tx=0x7fb77400c940 comp rx=0 tx=0).stop 2023-10-16T08:17:09.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 shutdown_connections 2023-10-16T08:17:09.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb7700664a0 0x7fb770068960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb78415cba0 0x7fb784147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7840d0210 0x7fb784149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 --2- 172.21.15.32:0/1219391618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7840b84e0 0x7fb7841572d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 >> 172.21.15.32:0/1219391618 conn(0x7fb7840719c0 msgr2=0x7fb7840b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 shutdown_connections 2023-10-16T08:17:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.246+0000 7fb78a8c9640 1 -- 172.21.15.32:0/1219391618 wait complete. 2023-10-16T08:17:09.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff19afd9640 1 Processor -- start 2023-10-16T08:17:09.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff19afd9640 1 -- start start 2023-10-16T08:17:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff19afd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940d0210 0x7ff1940d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff19afd9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940d1670 con 0x7ff1940b84e0 2023-10-16T08:17:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff19afd9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940d1950 con 0x7ff19415cba0 2023-10-16T08:17:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff19afd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940d1460 con 0x7ff1940d0210 2023-10-16T08:17:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff199576640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff1940b84e0 0x7ff1940b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37244/0 (socket says 172.21.15.32:37244) 2023-10-16T08:17:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff199576640 1 -- 172.21.15.32:0/629140506 learned_addr learned my addr 172.21.15.32:0/629140506 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff193fff640 1 --2- 172.21.15.32:0/629140506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940d0210 0x7ff1940d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587797182 0 0) 0x7ff1940d1670 con 0x7ff1940b84e0 2023-10-16T08:17:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff184003530 con 0x7ff1940b84e0 2023-10-16T08:17:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177007344 0 0) 0x7ff1940d1950 con 0x7ff19415cba0 2023-10-16T08:17:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1940d1670 con 0x7ff19415cba0 2023-10-16T08:17:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff193fff640 1 -- 172.21.15.32:0/629140506 >> v1:172.21.15.186:6789/0 conn(0x7ff19415cba0 legacy=0x7ff1940cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff193fff640 1 -- 172.21.15.32:0/629140506 >> v1:172.21.15.32:6789/0 conn(0x7ff1940b84e0 legacy=0x7ff1940b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.414+0000 7ff193fff640 1 -- 172.21.15.32:0/629140506 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940d1c30 con 0x7ff1940d0210 2023-10-16T08:17:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff193fff640 1 --2- 172.21.15.32:0/629140506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940d0210 0x7ff1940d0f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1800097f0 tx=0x7ff18001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf56ea93692cae4e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800aa620 con 0x7ff1940d0210 2023-10-16T08:17:09.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1800aa780 con 0x7ff1940d0210 2023-10-16T08:17:09.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff1937fe640 1 -- 172.21.15.32:0/629140506 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800aab10 con 0x7ff1940d0210 2023-10-16T08:17:09.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 -- 172.21.15.32:0/629140506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940d0210 msgr2=0x7ff1940d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 --2- 172.21.15.32:0/629140506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940d0210 0x7ff1940d0f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1800097f0 tx=0x7ff18001c4c0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 -- 172.21.15.32:0/629140506 shutdown_connections 2023-10-16T08:17:09.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 --2- 172.21.15.32:0/629140506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940d0210 0x7ff1940d0f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 -- 172.21.15.32:0/629140506 >> 172.21.15.32:0/629140506 conn(0x7ff1940719c0 msgr2=0x7ff194071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:09.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 -- 172.21.15.32:0/629140506 shutdown_connections 2023-10-16T08:17:09.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.418+0000 7ff19afd9640 1 -- 172.21.15.32:0/629140506 wait complete. 2023-10-16T08:17:09.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 Processor -- start 2023-10-16T08:17:09.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- start start 2023-10-16T08:17:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940b84e0 0x7ff19414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1940d0210 0x7ff194150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 0x7ff19415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff194151dd0 con 0x7ff19415cba0 2023-10-16T08:17:09.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940c7fd0 con 0x7ff1940d0210 2023-10-16T08:17:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff194151810 con 0x7ff1940b84e0 2023-10-16T08:17:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 0x7ff19415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 0x7ff19415dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39860/0 (socket says 172.21.15.32:39860) 2023-10-16T08:17:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 -- 172.21.15.32:0/2595114383 learned_addr learned my addr 172.21.15.32:0/2595114383 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff198d75640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940b84e0 0x7ff19414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff193fff640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1940d0210 0x7ff194150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 -- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940b84e0 msgr2=0x7ff19414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940b84e0 0x7ff19414c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 -- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1940d0210 msgr2=0x7ff194150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1940d0210 0x7ff194150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 -- 172.21.15.32:0/2595114383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009420 con 0x7ff19415cba0 2023-10-16T08:17:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff193fff640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1940d0210 0x7ff194150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff198d75640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940b84e0 0x7ff19414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff199576640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 0x7ff19415dea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00cbc0 tx=0x7ff18c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00ace0 con 0x7ff19415cba0 2023-10-16T08:17:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c00ae40 con 0x7ff19415cba0 2023-10-16T08:17:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff194151af0 con 0x7ff19415cba0 2023-10-16T08:17:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c012600 con 0x7ff19415cba0 2023-10-16T08:17:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19414c7b0 con 0x7ff19415cba0 2023-10-16T08:17:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.422+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1940c7ac0 con 0x7ff19415cba0 2023-10-16T08:17:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.426+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff18c012820 con 0x7ff19415cba0 2023-10-16T08:17:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.426+0000 7ff1917fa640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff1780664a0 0x7ff178068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.426+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff18c0841d0 con 0x7ff19415cba0 2023-10-16T08:17:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.426+0000 7ff198d75640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff1780664a0 0x7ff178068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.426+0000 7ff198d75640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff1780664a0 0x7ff178068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1880011f0 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.430+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1940c7ac0 con 0x7ff19415cba0 2023-10-16T08:17:09.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1940c7ac0 con 0x7ff19415cba0 2023-10-16T08:17:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff1917fa640 1 -- 172.21.15.32:0/2595114383 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1940c7ac0 con 0x7ff19415cba0 2023-10-16T08:17:09.592 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:09.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff1780664a0 msgr2=0x7ff178068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff1780664a0 0x7ff178068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1880011f0 tx=0x7ff188009040 comp rx=0 tx=0).stop 2023-10-16T08:17:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 msgr2=0x7ff19415dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 0x7ff19415dea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00cbc0 tx=0x7ff18c00a430 comp rx=0 tx=0).stop 2023-10-16T08:17:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 shutdown_connections 2023-10-16T08:17:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19415cba0 0x7ff19415dea0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1940d0210 0x7ff194150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff1780664a0 0x7ff178068960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 --2- 172.21.15.32:0/2595114383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff1940b84e0 0x7ff19414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 >> 172.21.15.32:0/2595114383 conn(0x7ff1940719c0 msgr2=0x7ff19415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 shutdown_connections 2023-10-16T08:17:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.586+0000 7ff19afd9640 1 -- 172.21.15.32:0/2595114383 wait complete. 2023-10-16T08:17:09.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 Processor -- start 2023-10-16T08:17:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 -- start start 2023-10-16T08:17:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec06a0f0 0x7f44ec0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44ec164dc0 0x7f44ec0ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec06c6d0 con 0x7f44ec164dc0 2023-10-16T08:17:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec06c950 con 0x7f44ec06a4d0 2023-10-16T08:17:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f3ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec1654f0 con 0x7f44ec06a0f0 2023-10-16T08:17:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44ec164dc0 0x7f44ec0ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44ec164dc0 0x7f44ec0ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39862/0 (socket says 172.21.15.32:39862) 2023-10-16T08:17:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 -- 172.21.15.32:0/3982035845 learned_addr learned my addr 172.21.15.32:0/3982035845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f1872640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec06a0f0 0x7f44ec0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f1071640 1 --2- 172.21.15.32:0/3982035845 >> v2:172.21.15.186:3300/0 conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 -- 172.21.15.32:0/3982035845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec06a0f0 msgr2=0x7f44ec0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec06a0f0 0x7f44ec0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 -- 172.21.15.32:0/3982035845 >> v2:172.21.15.186:3300/0 conn(0x7f44ec06a4d0 msgr2=0x7f44ec0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 --2- 172.21.15.32:0/3982035845 >> v2:172.21.15.186:3300/0 conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f2073640 1 -- 172.21.15.32:0/3982035845 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec1657d0 con 0x7f44ec164dc0 2023-10-16T08:17:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.754+0000 7f44f1872640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec06a0f0 0x7f44ec0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:09.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f2073640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec164dc0 0x7f44ec0ad6f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44ec071790 tx=0x7f44e401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4702cb96f593ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f0870640 1 -- 172.21.15.32:0/3982035845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40a57c0 con 0x7f44ec164dc0 2023-10-16T08:17:09.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f0870640 1 -- 172.21.15.32:0/3982035845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e40a5920 con 0x7f44ec164dc0 2023-10-16T08:17:09.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f0870640 1 -- 172.21.15.32:0/3982035845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40a5cb0 con 0x7f44ec164dc0 2023-10-16T08:17:09.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3982035845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec164dc0 msgr2=0x7f44ec0ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec164dc0 0x7f44ec0ad6f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44ec071790 tx=0x7f44e401c940 comp rx=0 tx=0).stop 2023-10-16T08:17:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3982035845 shutdown_connections 2023-10-16T08:17:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3982035845 >> v2:172.21.15.186:3300/0 conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec164dc0 0x7f44ec0ad6f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3982035845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec06a0f0 0x7f44ec0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3982035845 >> 172.21.15.32:0/3982035845 conn(0x7f44ec0b0da0 msgr2=0x7f44ec0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3982035845 shutdown_connections 2023-10-16T08:17:09.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.758+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3982035845 wait complete. 2023-10-16T08:17:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 Processor -- start 2023-10-16T08:17:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 -- start start 2023-10-16T08:17:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 0x7f44ec160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec164dc0 0x7f44ec0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0b1550 con 0x7f44ec06a0f0 2023-10-16T08:17:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0b1830 con 0x7f44ec06a4d0 2023-10-16T08:17:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0b1b10 con 0x7f44ec164dc0 2023-10-16T08:17:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 0x7f44ec160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 0x7f44ec160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39866/0 (socket says 172.21.15.32:39866) 2023-10-16T08:17:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 -- 172.21.15.32:0/3502492377 learned_addr learned my addr 172.21.15.32:0/3502492377 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1071640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f2073640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec164dc0 0x7f44ec0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 -- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec164dc0 msgr2=0x7f44ec0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec164dc0 0x7f44ec0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 -- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44ec06a4d0 msgr2=0x7f44ec0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 -- 172.21.15.32:0/3502492377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009b30 con 0x7f44ec06a0f0 2023-10-16T08:17:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1071640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f1872640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 0x7f44ec160c40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44e800e810 tx=0x7f44e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8004030 con 0x7f44ec06a0f0 2023-10-16T08:17:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e8004190 con 0x7f44ec06a0f0 2023-10-16T08:17:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8004520 con 0x7f44ec06a0f0 2023-10-16T08:17:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec0b1df0 con 0x7f44ec06a0f0 2023-10-16T08:17:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.762+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec161180 con 0x7f44ec06a0f0 2023-10-16T08:17:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.766+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec04e170 con 0x7f44ec06a0f0 2023-10-16T08:17:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.766+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f44e800c6d0 con 0x7f44ec06a0f0 2023-10-16T08:17:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.766+0000 7f44e27fc640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44d00664a0 0x7f44d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.766+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f44e8083ca0 con 0x7f44ec06a0f0 2023-10-16T08:17:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.766+0000 7f44f1071640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44d00664a0 0x7f44d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.766+0000 7f44f1071640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44d00664a0 0x7f44d0068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0ae8f0 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.770+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f44e8050ea0 con 0x7f44ec06a0f0 2023-10-16T08:17:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.930+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44ec0c5540 con 0x7f44ec06a0f0 2023-10-16T08:17:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.930+0000 7f44e27fc640 1 -- 172.21.15.32:0/3502492377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44e80544c0 con 0x7f44ec06a0f0 2023-10-16T08:17:09.938 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44d00664a0 msgr2=0x7f44d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44d00664a0 0x7f44d0068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0ae8f0 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2023-10-16T08:17:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 msgr2=0x7f44ec160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 0x7f44ec160c40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44e800e810 tx=0x7f44e800ecd0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 shutdown_connections 2023-10-16T08:17:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44d00664a0 0x7f44d0068960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44ec164dc0 0x7f44ec0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f44ec06a4d0 0x7f44ec0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 --2- 172.21.15.32:0/3502492377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44ec06a0f0 0x7f44ec160c40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 >> 172.21.15.32:0/3502492377 conn(0x7f44ec0b0da0 msgr2=0x7f44ec0afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 shutdown_connections 2023-10-16T08:17:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:09.934+0000 7f44f3ad6640 1 -- 172.21.15.32:0/3502492377 wait complete. 2023-10-16T08:17:09.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 Processor -- start 2023-10-16T08:17:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 -- start start 2023-10-16T08:17:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f621815cc60 0x7f62180cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6218161260 0x7f62180ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62180cc330 con 0x7f62180cebd0 2023-10-16T08:17:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62180cefe0 con 0x7f6218161260 2023-10-16T08:17:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f621e046640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62180cb720 con 0x7f621815cc60 2023-10-16T08:17:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f6216ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6218161260 0x7f62180ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f6216ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6218161260 0x7f62180ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39872/0 (socket says 172.21.15.32:39872) 2023-10-16T08:17:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f6216ffd640 1 -- 172.21.15.32:0/3983645906 learned_addr learned my addr 172.21.15.32:0/3983645906 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.098+0000 7f6217fff640 1 --1- 172.21.15.32:0/3983645906 >> v1:172.21.15.32:6789/0 conn(0x7f62180cebd0 0x7f62180b2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37254/0 (socket says 172.21.15.32:37254) 2023-10-16T08:17:10.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f6216ffd640 1 -- 172.21.15.32:0/3983645906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f621815cc60 msgr2=0x7f62180cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:10.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f62177fe640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f621815cc60 0x7f62180cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f6216ffd640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f621815cc60 0x7f62180cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f6216ffd640 1 -- 172.21.15.32:0/3983645906 >> v1:172.21.15.32:6789/0 conn(0x7f62180cebd0 legacy=0x7f62180b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f6216ffd640 1 -- 172.21.15.32:0/3983645906 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180cba00 con 0x7f6218161260 2023-10-16T08:17:10.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f62167fc640 1 -- 172.21.15.32:0/3983645906 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827571507 0 0) 0x7f62180cc330 con 0x7f62180cebd0 2023-10-16T08:17:10.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f62177fe640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f621815cc60 0x7f62180cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:10.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f6216ffd640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f62180ce690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f620c009870 tx=0x7f620c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1614b28afd1766 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f62167fc640 1 -- 172.21.15.32:0/3983645906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0aa670 con 0x7f6218161260 2023-10-16T08:17:10.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f62167fc640 1 -- 172.21.15.32:0/3983645906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c0aa7d0 con 0x7f6218161260 2023-10-16T08:17:10.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f62167fc640 1 -- 172.21.15.32:0/3983645906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0aab60 con 0x7f6218161260 2023-10-16T08:17:10.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f621e046640 1 -- 172.21.15.32:0/3983645906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 msgr2=0x7f62180ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f621e046640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f62180ce690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f620c009870 tx=0x7f620c01c510 comp rx=0 tx=0).stop 2023-10-16T08:17:10.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f621e046640 1 -- 172.21.15.32:0/3983645906 shutdown_connections 2023-10-16T08:17:10.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f621e046640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f62180ce690 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f621e046640 1 --2- 172.21.15.32:0/3983645906 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f621815cc60 0x7f62180cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.102+0000 7f621e046640 1 -- 172.21.15.32:0/3983645906 >> 172.21.15.32:0/3983645906 conn(0x7f621806e570 msgr2=0x7f621806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:10.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 -- 172.21.15.32:0/3983645906 shutdown_connections 2023-10-16T08:17:10.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 -- 172.21.15.32:0/3983645906 wait complete. 2023-10-16T08:17:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 Processor -- start 2023-10-16T08:17:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 -- start start 2023-10-16T08:17:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62180cebd0 0x7f6218154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621815cc60 0x7f621806b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f621806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621815f2d0 con 0x7f6218161260 2023-10-16T08:17:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6218160130 con 0x7f621815cc60 2023-10-16T08:17:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f621e046640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621815f520 con 0x7f62180cebd0 2023-10-16T08:17:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f621806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f621806b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39880/0 (socket says 172.21.15.32:39880) 2023-10-16T08:17:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 -- 172.21.15.32:0/2200779463 learned_addr learned my addr 172.21.15.32:0/2200779463 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f62177fe640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62180cebd0 0x7f6218154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6216ffd640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621815cc60 0x7f621806b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 -- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62180cebd0 msgr2=0x7f6218154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62180cebd0 0x7f6218154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 -- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621815cc60 msgr2=0x7f621806b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621815cc60 0x7f621806b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 -- 172.21.15.32:0/2200779463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0094d0 con 0x7f6218161260 2023-10-16T08:17:10.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f6217fff640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f621806b9d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f621000c810 tx=0x7f621000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.106+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000a640 con 0x7f6218161260 2023-10-16T08:17:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621000a7c0 con 0x7f6218161260 2023-10-16T08:17:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000ab70 con 0x7f6218161260 2023-10-16T08:17:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621815f800 con 0x7f6218161260 2023-10-16T08:17:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621815fdc0 con 0x7f6218161260 2023-10-16T08:17:10.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62180b3460 con 0x7f6218161260 2023-10-16T08:17:10.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f621000acd0 con 0x7f6218161260 2023-10-16T08:17:10.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f61fffff640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f62040664a0 0x7f6204068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6210083790 con 0x7f6218161260 2023-10-16T08:17:10.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.110+0000 7f62177fe640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f62040664a0 0x7f6204068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.114+0000 7f62177fe640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f62040664a0 0x7f6204068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f620800a9a0 tx=0x7f6208008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.114+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f62100507b0 con 0x7f6218161260 2023-10-16T08:17:10.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f621804e170 con 0x7f6218161260 2023-10-16T08:17:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f61fffff640 1 -- 172.21.15.32:0/2200779463 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6210018020 con 0x7f6218161260 2023-10-16T08:17:10.275 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f62040664a0 msgr2=0x7f6204068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f62040664a0 0x7f6204068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f620800a9a0 tx=0x7f6208008040 comp rx=0 tx=0).stop 2023-10-16T08:17:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 msgr2=0x7f621806b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f621806b9d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f621000c810 tx=0x7f621000ccd0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 shutdown_connections 2023-10-16T08:17:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6218161260 0x7f621806b9d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f621815cc60 0x7f621806b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f62040664a0 0x7f6204068960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 --2- 172.21.15.32:0/2200779463 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f62180cebd0 0x7f6218154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 >> 172.21.15.32:0/2200779463 conn(0x7f621806e570 msgr2=0x7f621815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 shutdown_connections 2023-10-16T08:17:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.270+0000 7f621e046640 1 -- 172.21.15.32:0/2200779463 wait complete. 2023-10-16T08:17:10.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.434+0000 7f27b6646640 1 Processor -- start 2023-10-16T08:17:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.434+0000 7f27b6646640 1 -- start start 2023-10-16T08:17:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b6646640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f27b015ca50 0x7f27b006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b6646640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ce30 0x7f27b00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b6646640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b01607e0 con 0x7f27b00b21c0 2023-10-16T08:17:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b6646640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b00cf060 con 0x7f27b015ca50 2023-10-16T08:17:10.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b6646640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0070210 con 0x7f27b015ce30 2023-10-16T08:17:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27affff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f27b015ca50 0x7f27b006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27af7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ce30 0x7f27b00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b4be3640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f27b00b21c0 0x7f27b0160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:59492/0 (socket says 172.21.15.32:59492) 2023-10-16T08:17:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27affff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f27b015ca50 0x7f27b006c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:48560/0 (socket says 172.21.15.32:48560) 2023-10-16T08:17:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27b4be3640 1 -- 172.21.15.32:0/213643728 learned_addr learned my addr 172.21.15.32:0/213643728 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27aeffd640 1 -- 172.21.15.32:0/213643728 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992763359 0 0) 0x7f27b01607e0 con 0x7f27b00b21c0 2023-10-16T08:17:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27aeffd640 1 -- 172.21.15.32:0/213643728 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2798003530 con 0x7f27b00b21c0 2023-10-16T08:17:10.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27affff640 1 -- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ce30 msgr2=0x7f27b00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27affff640 1 --2- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ce30 0x7f27b00cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27affff640 1 -- 172.21.15.32:0/213643728 >> v1:172.21.15.186:6789/0 conn(0x7f27b00b21c0 legacy=0x7f27b0160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27affff640 1 -- 172.21.15.32:0/213643728 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b00704f0 con 0x7f27b015ca50 2023-10-16T08:17:10.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.438+0000 7f27af7fe640 1 --2- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ce30 0x7f27b00cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:10.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27affff640 1 --2- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b015ca50 0x7f27b006c980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27a00097c0 tx=0x7f27a001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d42ae18d488f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27aeffd640 1 -- 172.21.15.32:0/213643728 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00ad660 con 0x7f27b015ca50 2023-10-16T08:17:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27aeffd640 1 -- 172.21.15.32:0/213643728 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a00ad7c0 con 0x7f27b015ca50 2023-10-16T08:17:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27aeffd640 1 -- 172.21.15.32:0/213643728 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00adb50 con 0x7f27b015ca50 2023-10-16T08:17:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 -- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b015ca50 msgr2=0x7f27b006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 --2- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b015ca50 0x7f27b006c980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27a00097c0 tx=0x7f27a001c450 comp rx=0 tx=0).stop 2023-10-16T08:17:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 -- 172.21.15.32:0/213643728 shutdown_connections 2023-10-16T08:17:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 --2- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ce30 0x7f27b00cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 --2- 172.21.15.32:0/213643728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b015ca50 0x7f27b006c980 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 -- 172.21.15.32:0/213643728 >> 172.21.15.32:0/213643728 conn(0x7f27b00cd1c0 msgr2=0x7f27b00cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 -- 172.21.15.32:0/213643728 shutdown_connections 2023-10-16T08:17:10.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 -- 172.21.15.32:0/213643728 wait complete. 2023-10-16T08:17:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 Processor -- start 2023-10-16T08:17:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.442+0000 7f27b6646640 1 -- start start 2023-10-16T08:17:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b00b21c0 0x7f27b00b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ca50 0x7f27b00b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 0x7f27b0156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0142870 con 0x7f27b015ce30 2023-10-16T08:17:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0153630 con 0x7f27b00b21c0 2023-10-16T08:17:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0153910 con 0x7f27b015ca50 2023-10-16T08:17:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 0x7f27b0156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27affff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b00b21c0 0x7f27b00b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 0x7f27b0156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39884/0 (socket says 172.21.15.32:39884) 2023-10-16T08:17:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 -- 172.21.15.32:0/900176620 learned_addr learned my addr 172.21.15.32:0/900176620 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27af7fe640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ca50 0x7f27b00b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 -- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ca50 msgr2=0x7f27b00b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ca50 0x7f27b00b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 -- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b00b21c0 msgr2=0x7f27b00b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b00b21c0 0x7f27b00b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 -- 172.21.15.32:0/900176620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a4009420 con 0x7f27b015ce30 2023-10-16T08:17:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b4be3640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 0x7f27b0156ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27a800cb40 tx=0x7f27a800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800aca0 con 0x7f27b015ce30 2023-10-16T08:17:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a800ae00 con 0x7f27b015ce30 2023-10-16T08:17:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a0009420 con 0x7f27b015ce30 2023-10-16T08:17:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8012620 con 0x7f27b015ce30 2023-10-16T08:17:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b00b2b60 con 0x7f27b015ce30 2023-10-16T08:17:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.446+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b006ce40 con 0x7f27b015ce30 2023-10-16T08:17:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.450+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27a8007070 con 0x7f27b015ce30 2023-10-16T08:17:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.450+0000 7f27acff9640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f279c0664a0 0x7f279c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.450+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27a8084290 con 0x7f27b015ce30 2023-10-16T08:17:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.450+0000 7f27affff640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f279c0664a0 0x7f279c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.450+0000 7f27affff640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f279c0664a0 0x7f279c068960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27a00af000 tx=0x7f27a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.454+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f27a8051370 con 0x7f27b015ce30 2023-10-16T08:17:10.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.610+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27b0074010 con 0x7f27b015ce30 2023-10-16T08:17:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.610+0000 7f27acff9640 1 -- 172.21.15.32:0/900176620 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f27a8054990 con 0x7f27b015ce30 2023-10-16T08:17:10.618 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:10.618 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f279c0664a0 msgr2=0x7f279c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f279c0664a0 0x7f279c068960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27a00af000 tx=0x7f27a0008040 comp rx=0 tx=0).stop 2023-10-16T08:17:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 msgr2=0x7f27b0156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 0x7f27b0156ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27a800cb40 tx=0x7f27a800a060 comp rx=0 tx=0).stop 2023-10-16T08:17:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 shutdown_connections 2023-10-16T08:17:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27b015ce30 0x7f27b0156ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f279c0664a0 0x7f279c068960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27b015ca50 0x7f27b00b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 --2- 172.21.15.32:0/900176620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27b00b21c0 0x7f27b00b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 >> 172.21.15.32:0/900176620 conn(0x7f27b00cd1c0 msgr2=0x7f27b00afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:10.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 shutdown_connections 2023-10-16T08:17:10.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.614+0000 7f27b6646640 1 -- 172.21.15.32:0/900176620 wait complete. 2023-10-16T08:17:10.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:10.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.782+0000 7f5061609640 1 Processor -- start 2023-10-16T08:17:10.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.782+0000 7f5061609640 1 -- start start 2023-10-16T08:17:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5061609640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c069e90 0x7f505c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5061609640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f505c06a270 0x7f505c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5061609640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c15ae30 con 0x7f505c0702f0 2023-10-16T08:17:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5061609640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0cb090 con 0x7f505c06a270 2023-10-16T08:17:10.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5061609640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0ca510 con 0x7f505c069e90 2023-10-16T08:17:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f505c0702f0 0x7f505c070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37266/0 (socket says 172.21.15.32:37266) 2023-10-16T08:17:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505b7fe640 1 -- 172.21.15.32:0/4075217173 learned_addr learned my addr 172.21.15.32:0/4075217173 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505affd640 1 --2- 172.21.15.32:0/4075217173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c069e90 0x7f505c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505a7fc640 1 --2- 172.21.15.32:0/4075217173 >> v2:172.21.15.186:3300/0 conn(0x7f505c06a270 0x7f505c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5059ffb640 1 -- 172.21.15.32:0/4075217173 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038104601 0 0) 0x7f505c15ae30 con 0x7f505c0702f0 2023-10-16T08:17:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f5059ffb640 1 -- 172.21.15.32:0/4075217173 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5044003530 con 0x7f505c0702f0 2023-10-16T08:17:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505affd640 1 -- 172.21.15.32:0/4075217173 >> v2:172.21.15.186:3300/0 conn(0x7f505c06a270 msgr2=0x7f505c0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505affd640 1 --2- 172.21.15.32:0/4075217173 >> v2:172.21.15.186:3300/0 conn(0x7f505c06a270 0x7f505c0c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505affd640 1 -- 172.21.15.32:0/4075217173 >> v1:172.21.15.32:6789/0 conn(0x7f505c0702f0 legacy=0x7f505c070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.786+0000 7f505affd640 1 -- 172.21.15.32:0/4075217173 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0ca7f0 con 0x7f505c069e90 2023-10-16T08:17:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f505affd640 1 --2- 172.21.15.32:0/4075217173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c069e90 0x7f505c06d010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f504c0097c0 tx=0x7f504c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc9bd1bcc1963b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5059ffb640 1 -- 172.21.15.32:0/4075217173 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0ab650 con 0x7f505c069e90 2023-10-16T08:17:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5059ffb640 1 -- 172.21.15.32:0/4075217173 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c0ab7b0 con 0x7f505c069e90 2023-10-16T08:17:10.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5059ffb640 1 -- 172.21.15.32:0/4075217173 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0abb40 con 0x7f505c069e90 2023-10-16T08:17:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- 172.21.15.32:0/4075217173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c069e90 msgr2=0x7f505c06d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 --2- 172.21.15.32:0/4075217173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c069e90 0x7f505c06d010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f504c0097c0 tx=0x7f504c01c450 comp rx=0 tx=0).stop 2023-10-16T08:17:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- 172.21.15.32:0/4075217173 shutdown_connections 2023-10-16T08:17:10.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 --2- 172.21.15.32:0/4075217173 >> v2:172.21.15.186:3300/0 conn(0x7f505c06a270 0x7f505c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 --2- 172.21.15.32:0/4075217173 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c069e90 0x7f505c06d010 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- 172.21.15.32:0/4075217173 >> 172.21.15.32:0/4075217173 conn(0x7f505c0b0da0 msgr2=0x7f505c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:10.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- 172.21.15.32:0/4075217173 shutdown_connections 2023-10-16T08:17:10.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- 172.21.15.32:0/4075217173 wait complete. 2023-10-16T08:17:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 Processor -- start 2023-10-16T08:17:10.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- start start 2023-10-16T08:17:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f505c069e90 0x7f505c0ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 0x7f505c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c0702f0 0x7f505c1545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15ca40 con 0x7f505c06a270 2023-10-16T08:17:10.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15bdc0 con 0x7f505c069e90 2023-10-16T08:17:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.790+0000 7f5061609640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15c0a0 con 0x7f505c0702f0 2023-10-16T08:17:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 0x7f505c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 0x7f505c1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39888/0 (socket says 172.21.15.32:39888) 2023-10-16T08:17:10.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 -- 172.21.15.32:0/2691657031 learned_addr learned my addr 172.21.15.32:0/2691657031 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505affd640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f505c069e90 0x7f505c0ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505b7fe640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c0702f0 0x7f505c1545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 -- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c0702f0 msgr2=0x7f505c1545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c0702f0 0x7f505c1545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 -- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f505c069e90 msgr2=0x7f505c0ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f505c069e90 0x7f505c0ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 -- 172.21.15.32:0/2691657031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c009420 con 0x7f505c06a270 2023-10-16T08:17:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f505a7fc640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 0x7f505c1570d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f505000e930 tx=0x7f505000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050009690 con 0x7f505c06a270 2023-10-16T08:17:10.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000ca00 con 0x7f505c06a270 2023-10-16T08:17:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c15c380 con 0x7f505c06a270 2023-10-16T08:17:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000cd70 con 0x7f505c06a270 2023-10-16T08:17:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c15ba20 con 0x7f505c06a270 2023-10-16T08:17:10.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c04e170 con 0x7f505c06a270 2023-10-16T08:17:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5050013470 con 0x7f505c06a270 2023-10-16T08:17:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.794+0000 7f50437fe640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5048066470 0x7f5048068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.798+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5050084470 con 0x7f505c06a270 2023-10-16T08:17:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.798+0000 7f505affd640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5048066470 0x7f5048068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.798+0000 7f505affd640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5048066470 0x7f5048068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f504c0af000 tx=0x7f504c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.802+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5050051690 con 0x7f505c06a270 2023-10-16T08:17:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.958+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f505c0c2090 con 0x7f505c06a270 2023-10-16T08:17:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.958+0000 7f50437fe640 1 -- 172.21.15.32:0/2691657031 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5050054cb0 con 0x7f505c06a270 2023-10-16T08:17:10.965 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5048066470 msgr2=0x7f5048068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5048066470 0x7f5048068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f504c0af000 tx=0x7f504c0a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 msgr2=0x7f505c1570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 0x7f505c1570d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f505000e930 tx=0x7f505000edf0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 shutdown_connections 2023-10-16T08:17:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5048066470 0x7f5048068930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f505c0702f0 0x7f505c1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c06a270 0x7f505c1570d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 --2- 172.21.15.32:0/2691657031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f505c069e90 0x7f505c0ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:10.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 >> 172.21.15.32:0/2691657031 conn(0x7f505c0b0da0 msgr2=0x7f505c0af940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:10.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 shutdown_connections 2023-10-16T08:17:10.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:10.962+0000 7f5061609640 1 -- 172.21.15.32:0/2691657031 wait complete. 2023-10-16T08:17:10.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.126+0000 7f68f1ed6640 1 Processor -- start 2023-10-16T08:17:11.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68f1ed6640 1 -- start start 2023-10-16T08:17:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68f1ed6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f68ec0d0210 0x7f68ec0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68f1ed6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0d1670 con 0x7f68ec0b84e0 2023-10-16T08:17:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68f1ed6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0d1950 con 0x7f68ec15cba0 2023-10-16T08:17:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68f1ed6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ec0d1460 con 0x7f68ec0d0210 2023-10-16T08:17:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ebfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68ec0b84e0 0x7f68ec0b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37272/0 (socket says 172.21.15.32:37272) 2023-10-16T08:17:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ebfff640 1 -- 172.21.15.32:0/594525033 learned_addr learned my addr 172.21.15.32:0/594525033 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 --2- 172.21.15.32:0/594525033 >> v2:172.21.15.186:3300/0 conn(0x7f68ec0d0210 0x7f68ec0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552309 0 0) 0x7f68ec0d1670 con 0x7f68ec0b84e0 2023-10-16T08:17:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d4003530 con 0x7f68ec0b84e0 2023-10-16T08:17:11.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2174777676 0 0) 0x7f68d4003530 con 0x7f68ec0b84e0 2023-10-16T08:17:11.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 >> v2:172.21.15.186:3300/0 conn(0x7f68ec0d0210 msgr2=0x7f68ec0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 --2- 172.21.15.32:0/594525033 >> v2:172.21.15.186:3300/0 conn(0x7f68ec0d0210 0x7f68ec0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 >> v1:172.21.15.186:6789/0 conn(0x7f68ec15cba0 legacy=0x7f68ec0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0d1c30 con 0x7f68ec0b84e0 2023-10-16T08:17:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f68e4001a00 con 0x7f68ec0b84e0 2023-10-16T08:17:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 >> v1:172.21.15.32:6789/0 conn(0x7f68ec0b84e0 legacy=0x7f68ec0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d4003980 0x7f68d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68d4005190 0x7f68d400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0d1460 con 0x7f68d40047d0 2023-10-16T08:17:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d400c130 con 0x7f68d4003980 2023-10-16T08:17:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d400c2f0 con 0x7f68d4005190 2023-10-16T08:17:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 --2- 172.21.15.32:0/594525033 >> v2:172.21.15.186:3300/0 conn(0x7f68ec0d0210 0x7f68ec0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eb7fe640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d4003980 0x7f68d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ebfff640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68d4005190 0x7f68d400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 -- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68d4005190 msgr2=0x7f68d400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:11.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68d4005190 0x7f68d400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 -- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d4003980 msgr2=0x7f68d4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:11.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68d4003980 0x7f68d4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68eaffd640 1 -- 172.21.15.32:0/594525033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4003530 con 0x7f68d40047d0 2023-10-16T08:17:11.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.130+0000 7f68ebfff640 1 -- 172.21.15.32:0/594525033 reap_dead start 2023-10-16T08:17:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68eaffd640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68d4004c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009c80 tx=0x7f68e001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e057df89d17048 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00a5900 con 0x7f68d40047d0 2023-10-16T08:17:11.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e00a5a60 con 0x7f68d40047d0 2023-10-16T08:17:11.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68ea7fc640 1 -- 172.21.15.32:0/594525033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00a5df0 con 0x7f68d40047d0 2023-10-16T08:17:11.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 -- 172.21.15.32:0/594525033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 msgr2=0x7f68d4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68d4004c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009c80 tx=0x7f68e001cbb0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 -- 172.21.15.32:0/594525033 shutdown_connections 2023-10-16T08:17:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/594525033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68d4004c50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 -- 172.21.15.32:0/594525033 >> 172.21.15.32:0/594525033 conn(0x7f68ec0719c0 msgr2=0x7f68ec071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 -- 172.21.15.32:0/594525033 shutdown_connections 2023-10-16T08:17:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.134+0000 7f68f1ed6640 1 -- 172.21.15.32:0/594525033 wait complete. 2023-10-16T08:17:11.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 Processor -- start 2023-10-16T08:17:11.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 -- start start 2023-10-16T08:17:11.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68ec15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68ec0c73b0 0x7f68ec151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68ec0cb070 0x7f68ec0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec1544f0 con 0x7f68d40047d0 2023-10-16T08:17:11.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0c9c20 con 0x7f68ec0c73b0 2023-10-16T08:17:11.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec0c7840 con 0x7f68ec0cb070 2023-10-16T08:17:11.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68ec15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68ec15fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39908/0 (socket says 172.21.15.32:39908) 2023-10-16T08:17:11.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 -- 172.21.15.32:0/4017988088 learned_addr learned my addr 172.21.15.32:0/4017988088 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:11.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eaffd640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68ec0c73b0 0x7f68ec151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68ebfff640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68ec0cb070 0x7f68ec0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 -- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68ec0cb070 msgr2=0x7f68ec0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68ec0cb070 0x7f68ec0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 -- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68ec0c73b0 msgr2=0x7f68ec151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68ec0c73b0 0x7f68ec151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 -- 172.21.15.32:0/4017988088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0c7b20 con 0x7f68d40047d0 2023-10-16T08:17:11.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eaffd640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68ec0c73b0 0x7f68ec151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:11.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68eb7fe640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68ec15fad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00f770 tx=0x7f68dc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc001350 con 0x7f68d40047d0 2023-10-16T08:17:11.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc0050d0 con 0x7f68d40047d0 2023-10-16T08:17:11.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc005480 con 0x7f68d40047d0 2023-10-16T08:17:11.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ec0c7e00 con 0x7f68d40047d0 2023-10-16T08:17:11.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.138+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ec160010 con 0x7f68d40047d0 2023-10-16T08:17:11.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.142+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ec0c7ff0 con 0x7f68d40047d0 2023-10-16T08:17:11.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.142+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68dc00d6d0 con 0x7f68d40047d0 2023-10-16T08:17:11.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.142+0000 7f68d3fff640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68d80664f0 0x7f68d80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.142+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f68dc0840e0 con 0x7f68d40047d0 2023-10-16T08:17:11.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.142+0000 7f68eaffd640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68d80664f0 0x7f68d80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.142+0000 7f68eaffd640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68d80664f0 0x7f68d80689b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68e00af000 tx=0x7f68e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.146+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68dc051180 con 0x7f68d40047d0 2023-10-16T08:17:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68ec04e170 con 0x7f68d40047d0 2023-10-16T08:17:11.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68d3fff640 1 -- 172.21.15.32:0/4017988088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68dc0547a0 con 0x7f68d40047d0 2023-10-16T08:17:11.308 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:11.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68d80664f0 msgr2=0x7f68d80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68d80664f0 0x7f68d80689b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68e00af000 tx=0x7f68e0008040 comp rx=0 tx=0).stop 2023-10-16T08:17:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 msgr2=0x7f68ec15fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68ec15fad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00f770 tx=0x7f68dc00fc30 comp rx=0 tx=0).stop 2023-10-16T08:17:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 shutdown_connections 2023-10-16T08:17:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68d80664f0 0x7f68d80689b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68ec0cb070 0x7f68ec0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68ec0c73b0 0x7f68ec151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 --2- 172.21.15.32:0/4017988088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d40047d0 0x7f68ec15fad0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 >> 172.21.15.32:0/4017988088 conn(0x7f68ec0719c0 msgr2=0x7f68ec1596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.302+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 shutdown_connections 2023-10-16T08:17:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.306+0000 7f68f1ed6640 1 -- 172.21.15.32:0/4017988088 wait complete. 2023-10-16T08:17:11.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 Processor -- start 2023-10-16T08:17:11.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 -- start start 2023-10-16T08:17:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1f6015cba0 0x7f1f600d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f600d0210 0x7f1f600b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f600d1670 con 0x7f1f600b84e0 2023-10-16T08:17:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600d1460 con 0x7f1f600d0210 2023-10-16T08:17:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f67ed6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600d1950 con 0x7f1f6015cba0 2023-10-16T08:17:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f600d0210 0x7f1f600b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f600d0210 0x7f1f600b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39922/0 (socket says 172.21.15.32:39922) 2023-10-16T08:17:11.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f65471640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f600b84e0 0x7f1f600cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37286/0 (socket says 172.21.15.32:37286) 2023-10-16T08:17:11.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 -- 172.21.15.32:0/3187879039 learned_addr learned my addr 172.21.15.32:0/3187879039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:11.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f65c72640 1 --2- 172.21.15.32:0/3187879039 >> v2:172.21.15.186:3300/0 conn(0x7f1f6015cba0 0x7f1f600d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 -- 172.21.15.32:0/3187879039 >> v2:172.21.15.186:3300/0 conn(0x7f1f6015cba0 msgr2=0x7f1f600d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 --2- 172.21.15.32:0/3187879039 >> v2:172.21.15.186:3300/0 conn(0x7f1f6015cba0 0x7f1f600d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 -- 172.21.15.32:0/3187879039 >> v1:172.21.15.32:6789/0 conn(0x7f1f600b84e0 legacy=0x7f1f600cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f66473640 1 -- 172.21.15.32:0/3187879039 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600d1c30 con 0x7f1f600d0210 2023-10-16T08:17:11.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.470+0000 7f1f64c70640 1 -- 172.21.15.32:0/3187879039 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694205944 0 0) 0x7f1f600d1670 con 0x7f1f600b84e0 2023-10-16T08:17:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f66473640 1 --2- 172.21.15.32:0/3187879039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f600b5cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f600b4890 tx=0x7f1f5801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8c2722d68daee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f64c70640 1 -- 172.21.15.32:0/3187879039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580a5800 con 0x7f1f600d0210 2023-10-16T08:17:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f64c70640 1 -- 172.21.15.32:0/3187879039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f580a5960 con 0x7f1f600d0210 2023-10-16T08:17:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f64c70640 1 -- 172.21.15.32:0/3187879039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580a5cf0 con 0x7f1f600d0210 2023-10-16T08:17:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 -- 172.21.15.32:0/3187879039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 msgr2=0x7f1f600b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/3187879039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f600b5cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f600b4890 tx=0x7f1f5801c940 comp rx=0 tx=0).stop 2023-10-16T08:17:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 -- 172.21.15.32:0/3187879039 shutdown_connections 2023-10-16T08:17:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/3187879039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f600b5cf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/3187879039 >> v2:172.21.15.186:3300/0 conn(0x7f1f6015cba0 0x7f1f600d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 -- 172.21.15.32:0/3187879039 >> 172.21.15.32:0/3187879039 conn(0x7f1f600719c0 msgr2=0x7f1f60071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 -- 172.21.15.32:0/3187879039 shutdown_connections 2023-10-16T08:17:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.474+0000 7f1f67ed6640 1 -- 172.21.15.32:0/3187879039 wait complete. 2023-10-16T08:17:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 Processor -- start 2023-10-16T08:17:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 -- start start 2023-10-16T08:17:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600b84e0 0x7f1f6015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f6014b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6015cba0 0x7f1f6014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60156770 con 0x7f1f600d0210 2023-10-16T08:17:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60156a50 con 0x7f1f600b84e0 2023-10-16T08:17:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6014bbf0 con 0x7f1f6015cba0 2023-10-16T08:17:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f6014b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f6014b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39924/0 (socket says 172.21.15.32:39924) 2023-10-16T08:17:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 -- 172.21.15.32:0/2347523133 learned_addr learned my addr 172.21.15.32:0/2347523133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 -- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6015cba0 msgr2=0x7f1f6014dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6015cba0 0x7f1f6014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 -- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600b84e0 msgr2=0x7f1f6015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600b84e0 0x7f1f6015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 -- 172.21.15.32:0/2347523133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f58009b30 con 0x7f1f600d0210 2023-10-16T08:17:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f65471640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f6014b310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f500011f0 tx=0x7f1f5000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500074e0 con 0x7f1f600d0210 2023-10-16T08:17:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f50007640 con 0x7f1f600d0210 2023-10-16T08:17:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50004a10 con 0x7f1f600d0210 2023-10-16T08:17:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6014bed0 con 0x7f1f600d0210 2023-10-16T08:17:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.478+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6015ac10 con 0x7f1f600d0210 2023-10-16T08:17:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.482+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6014c0c0 con 0x7f1f600d0210 2023-10-16T08:17:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.482+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1f50004b70 con 0x7f1f600d0210 2023-10-16T08:17:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.482+0000 7f1f567fc640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1f440664a0 0x7f1f44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.482+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f50083550 con 0x7f1f600d0210 2023-10-16T08:17:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.482+0000 7f1f65c72640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1f440664a0 0x7f1f44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.482+0000 7f1f65c72640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1f440664a0 0x7f1f44068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006aa0 tx=0x7f1f5c006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.486+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1f50050600 con 0x7f1f600d0210 2023-10-16T08:17:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.642+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f6004e170 con 0x7f1f600d0210 2023-10-16T08:17:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.642+0000 7f1f567fc640 1 -- 172.21.15.32:0/2347523133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f500180a0 con 0x7f1f600d0210 2023-10-16T08:17:11.649 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1f440664a0 msgr2=0x7f1f44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1f440664a0 0x7f1f44068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006aa0 tx=0x7f1f5c006a30 comp rx=0 tx=0).stop 2023-10-16T08:17:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 msgr2=0x7f1f6014b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f6014b310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f500011f0 tx=0x7f1f5000c9b0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 shutdown_connections 2023-10-16T08:17:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1f440664a0 0x7f1f44068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1f6015cba0 0x7f1f6014dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f600d0210 0x7f1f6014b310 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 --2- 172.21.15.32:0/2347523133 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f600b84e0 0x7f1f6015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 >> 172.21.15.32:0/2347523133 conn(0x7f1f600719c0 msgr2=0x7f1f60153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:11.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 shutdown_connections 2023-10-16T08:17:11.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.646+0000 7f1f67ed6640 1 -- 172.21.15.32:0/2347523133 wait complete. 2023-10-16T08:17:11.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.810+0000 7ff770609640 1 Processor -- start 2023-10-16T08:17:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.810+0000 7ff770609640 1 -- start start 2023-10-16T08:17:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff770609640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7680b84e0 0x7ff7680d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff770609640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680d1670 con 0x7ff7680d0210 2023-10-16T08:17:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff770609640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680d1950 con 0x7ff76815cba0 2023-10-16T08:17:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff770609640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680d1460 con 0x7ff7680b84e0 2023-10-16T08:17:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76eba6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7680b84e0 0x7ff7680d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76dba4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7680d0210 0x7ff7680b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37292/0 (socket says 172.21.15.32:37292) 2023-10-16T08:17:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76dba4640 1 -- 172.21.15.32:0/2029615941 learned_addr learned my addr 172.21.15.32:0/2029615941 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76eba6640 1 -- 172.21.15.32:0/2029615941 >> v1:172.21.15.186:6789/0 conn(0x7ff76815cba0 legacy=0x7ff7680cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76eba6640 1 -- 172.21.15.32:0/2029615941 >> v1:172.21.15.32:6789/0 conn(0x7ff7680d0210 legacy=0x7ff7680b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76eba6640 1 -- 172.21.15.32:0/2029615941 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680d1c30 con 0x7ff7680b84e0 2023-10-16T08:17:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.814+0000 7ff76d3a3640 1 -- 172.21.15.32:0/2029615941 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192291506 0 0) 0x7ff7680d1670 con 0x7ff7680d0210 2023-10-16T08:17:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff76eba6640 1 --2- 172.21.15.32:0/2029615941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680b84e0 0x7ff7680d0f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff76001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ca15aa7383e13c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff76d3a3640 1 -- 172.21.15.32:0/2029615941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600a5940 con 0x7ff7680b84e0 2023-10-16T08:17:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff76d3a3640 1 -- 172.21.15.32:0/2029615941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7600a5aa0 con 0x7ff7680b84e0 2023-10-16T08:17:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff76d3a3640 1 -- 172.21.15.32:0/2029615941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600a5e30 con 0x7ff7680b84e0 2023-10-16T08:17:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 -- 172.21.15.32:0/2029615941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680b84e0 msgr2=0x7ff7680d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 --2- 172.21.15.32:0/2029615941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680b84e0 0x7ff7680d0f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff76001c880 comp rx=0 tx=0).stop 2023-10-16T08:17:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 -- 172.21.15.32:0/2029615941 shutdown_connections 2023-10-16T08:17:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 --2- 172.21.15.32:0/2029615941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680b84e0 0x7ff7680d0f20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 -- 172.21.15.32:0/2029615941 >> 172.21.15.32:0/2029615941 conn(0x7ff7680719c0 msgr2=0x7ff768071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 -- 172.21.15.32:0/2029615941 shutdown_connections 2023-10-16T08:17:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 -- 172.21.15.32:0/2029615941 wait complete. 2023-10-16T08:17:11.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 Processor -- start 2023-10-16T08:17:11.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.818+0000 7ff770609640 1 -- start start 2023-10-16T08:17:11.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7680b84e0 0x7ff76815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7680d0210 0x7ff768158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 0x7ff768149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff768159710 con 0x7ff76815cba0 2023-10-16T08:17:11.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76815aa80 con 0x7ff7680b84e0 2023-10-16T08:17:11.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76814c3a0 con 0x7ff7680d0210 2023-10-16T08:17:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 0x7ff768149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 0x7ff768149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39942/0 (socket says 172.21.15.32:39942) 2023-10-16T08:17:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 -- 172.21.15.32:0/3120306713 learned_addr learned my addr 172.21.15.32:0/3120306713 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76dba4640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7680d0210 0x7ff768158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76e3a5640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7680b84e0 0x7ff76815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 -- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7680d0210 msgr2=0x7ff768158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7680d0210 0x7ff768158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 -- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7680b84e0 msgr2=0x7ff76815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7680b84e0 0x7ff76815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:11.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 -- 172.21.15.32:0/3120306713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff760009a70 con 0x7ff76815cba0 2023-10-16T08:17:11.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76e3a5640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7680b84e0 0x7ff76815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:11.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff76eba6640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 0x7ff768149260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff760019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600ade40 con 0x7ff76815cba0 2023-10-16T08:17:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7600b7410 con 0x7ff76815cba0 2023-10-16T08:17:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600b77c0 con 0x7ff76815cba0 2023-10-16T08:17:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff768158b80 con 0x7ff76815cba0 2023-10-16T08:17:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff768159a60 con 0x7ff76815cba0 2023-10-16T08:17:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7681598c0 con 0x7ff76815cba0 2023-10-16T08:17:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.822+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7600b4050 con 0x7ff76815cba0 2023-10-16T08:17:11.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.826+0000 7ff75effd640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff74c0664a0 0x7ff74c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:11.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.826+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff76011f5a0 con 0x7ff76815cba0 2023-10-16T08:17:11.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.826+0000 7ff76e3a5640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff74c0664a0 0x7ff74c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:11.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.826+0000 7ff76e3a5640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff74c0664a0 0x7ff74c068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7681602f0 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:11.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.830+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff7681598c0 con 0x7ff76815cba0 2023-10-16T08:17:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff7681598c0 con 0x7ff76815cba0 2023-10-16T08:17:11.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff75effd640 1 -- 172.21.15.32:0/3120306713 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff7681598c0 con 0x7ff76815cba0 2023-10-16T08:17:11.996 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff74c0664a0 msgr2=0x7ff74c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff74c0664a0 0x7ff74c068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7681602f0 tx=0x7ff764008040 comp rx=0 tx=0).stop 2023-10-16T08:17:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 msgr2=0x7ff768149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 0x7ff768149260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff760019620 comp rx=0 tx=0).stop 2023-10-16T08:17:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 shutdown_connections 2023-10-16T08:17:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff76815cba0 0x7ff768149260 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff74c0664a0 0x7ff74c068960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff7680d0210 0x7ff768158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 --2- 172.21.15.32:0/3120306713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7680b84e0 0x7ff76815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 >> 172.21.15.32:0/3120306713 conn(0x7ff7680719c0 msgr2=0x7ff768072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 shutdown_connections 2023-10-16T08:17:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:11.990+0000 7ff770609640 1 -- 172.21.15.32:0/3120306713 wait complete. 2023-10-16T08:17:12.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:12.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.154+0000 7f2bd75f1640 1 Processor -- start 2023-10-16T08:17:12.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.154+0000 7f2bd75f1640 1 -- start start 2023-10-16T08:17:12.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd75f1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2bd0070c40 0x7f2bd015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd75f1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bd00b4020 0x7f2bd00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd75f1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd015be60 con 0x7f2bd00b0620 2023-10-16T08:17:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd75f1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd015b3b0 con 0x7f2bd00b4020 2023-10-16T08:17:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd75f1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd015c1f0 con 0x7f2bd0070c40 2023-10-16T08:17:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bd00b4020 0x7f2bd00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bd00b4020 0x7f2bd00b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39952/0 (socket says 172.21.15.32:39952) 2023-10-16T08:17:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/1086371302 learned_addr learned my addr 172.21.15.32:0/1086371302 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:12.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd538d640 1 --2- 172.21.15.32:0/1086371302 >> v2:172.21.15.186:3300/0 conn(0x7f2bd0070c40 0x7f2bd015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/1086371302 >> v2:172.21.15.186:3300/0 conn(0x7f2bd0070c40 msgr2=0x7f2bd015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:12.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 --2- 172.21.15.32:0/1086371302 >> v2:172.21.15.186:3300/0 conn(0x7f2bd0070c40 0x7f2bd015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/1086371302 >> v1:172.21.15.186:6789/0 conn(0x7f2bd00b0620 legacy=0x7f2bd015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.158+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/1086371302 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00b0a30 con 0x7f2bd00b4020 2023-10-16T08:17:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd4b8c640 1 --2- 172.21.15.32:0/1086371302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b4020 0x7f2bd00b00e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2bd006ae80 tx=0x7f2bc001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd71e9fa8dcd7b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bc7fff640 1 -- 172.21.15.32:0/1086371302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00ab670 con 0x7f2bd00b4020 2023-10-16T08:17:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bc7fff640 1 -- 172.21.15.32:0/1086371302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00ab7d0 con 0x7f2bd00b4020 2023-10-16T08:17:12.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bc7fff640 1 -- 172.21.15.32:0/1086371302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00abb60 con 0x7f2bd00b4020 2023-10-16T08:17:12.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 -- 172.21.15.32:0/1086371302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b4020 msgr2=0x7f2bd00b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/1086371302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b4020 0x7f2bd00b00e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2bd006ae80 tx=0x7f2bc001c470 comp rx=0 tx=0).stop 2023-10-16T08:17:12.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 -- 172.21.15.32:0/1086371302 shutdown_connections 2023-10-16T08:17:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/1086371302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b4020 0x7f2bd00b00e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/1086371302 >> v2:172.21.15.186:3300/0 conn(0x7f2bd0070c40 0x7f2bd015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 -- 172.21.15.32:0/1086371302 >> 172.21.15.32:0/1086371302 conn(0x7f2bd006c190 msgr2=0x7f2bd006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 -- 172.21.15.32:0/1086371302 shutdown_connections 2023-10-16T08:17:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 -- 172.21.15.32:0/1086371302 wait complete. 2023-10-16T08:17:12.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 Processor -- start 2023-10-16T08:17:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.162+0000 7f2bd75f1640 1 -- start start 2023-10-16T08:17:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bd0070c40 0x7f2bd01447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 0x7f2bd00c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bd00b4020 0x7f2bd0142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd014d470 con 0x7f2bd00b0620 2023-10-16T08:17:12.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd014d750 con 0x7f2bd00b4020 2023-10-16T08:17:12.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd014da30 con 0x7f2bd0070c40 2023-10-16T08:17:12.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 0x7f2bd00c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 0x7f2bd00c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39962/0 (socket says 172.21.15.32:39962) 2023-10-16T08:17:12.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/654696721 learned_addr learned my addr 172.21.15.32:0/654696721 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd538d640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bd0070c40 0x7f2bd01447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd5b8e640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bd00b4020 0x7f2bd0142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bd0070c40 msgr2=0x7f2bd01447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bd0070c40 0x7f2bd01447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bd00b4020 msgr2=0x7f2bd0142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bd00b4020 0x7f2bd0142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 -- 172.21.15.32:0/654696721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00094d0 con 0x7f2bd00b0620 2023-10-16T08:17:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd4b8c640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 0x7f2bd00c23f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0019080 tx=0x7f2bc001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00b4800 con 0x7f2bd00b0620 2023-10-16T08:17:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00b4960 con 0x7f2bd00b0620 2023-10-16T08:17:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd014dd10 con 0x7f2bd00b0620 2023-10-16T08:17:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00b4d10 con 0x7f2bd00b0620 2023-10-16T08:17:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd014d090 con 0x7f2bd00b0620 2023-10-16T08:17:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0074010 con 0x7f2bd00b0620 2023-10-16T08:17:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.166+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2bc00be3f0 con 0x7f2bd00b0620 2023-10-16T08:17:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.170+0000 7f2bc5ffb640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bb40664a0 0x7f2bb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.170+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2bc011ecb0 con 0x7f2bd00b0620 2023-10-16T08:17:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.170+0000 7f2bd538d640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bb40664a0 0x7f2bb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.170+0000 7f2bd538d640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bb40664a0 0x7f2bb4068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006aa0 tx=0x7f2bcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.174+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2bc00ebfd0 con 0x7f2bd00b0620 2023-10-16T08:17:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bd00b1280 con 0x7f2bd00b0620 2023-10-16T08:17:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bc5ffb640 1 -- 172.21.15.32:0/654696721 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f2bc00ef5f0 con 0x7f2bd00b0620 2023-10-16T08:17:12.340 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:12.340 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:12.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bb40664a0 msgr2=0x7f2bb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bb40664a0 0x7f2bb4068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006aa0 tx=0x7f2bcc008040 comp rx=0 tx=0).stop 2023-10-16T08:17:12.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 msgr2=0x7f2bd00c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 0x7f2bd00c23f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0019080 tx=0x7f2bc001acb0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 shutdown_connections 2023-10-16T08:17:12.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bd00b4020 0x7f2bd0142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd00b0620 0x7f2bd00c23f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bb40664a0 0x7f2bb4068960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 --2- 172.21.15.32:0/654696721 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2bd0070c40 0x7f2bd01447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 >> 172.21.15.32:0/654696721 conn(0x7f2bd006c190 msgr2=0x7f2bd00aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:12.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 shutdown_connections 2023-10-16T08:17:12.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.334+0000 7f2bd75f1640 1 -- 172.21.15.32:0/654696721 wait complete. 2023-10-16T08:17:12.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:12.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:12.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.502+0000 7fbcb3b3e640 1 Processor -- start 2023-10-16T08:17:12.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.502+0000 7fbcb3b3e640 1 -- start start 2023-10-16T08:17:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb3b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac069e90 0x7fbcac06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb3b3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbcac06a270 0x7fbcac0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb3b3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac15ae30 con 0x7fbcac0702f0 2023-10-16T08:17:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb3b3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac0cb090 con 0x7fbcac06a270 2023-10-16T08:17:12.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb3b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ca510 con 0x7fbcac069e90 2023-10-16T08:17:12.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb18da640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac069e90 0x7fbcac06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbcac06a270 0x7fbcac0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbcac06a270 0x7fbcac0c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:48690/0 (socket says 172.21.15.32:48690) 2023-10-16T08:17:12.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 -- 172.21.15.32:0/3835828627 learned_addr learned my addr 172.21.15.32:0/3835828627 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:12.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb20db640 1 --1- 172.21.15.32:0/3835828627 >> v1:172.21.15.186:6789/0 conn(0x7fbcac0702f0 0x7fbcac070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:59538/0 (socket says 172.21.15.32:59538) 2023-10-16T08:17:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 -- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac069e90 msgr2=0x7fbcac06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 --2- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac069e90 0x7fbcac06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 -- 172.21.15.32:0/3835828627 >> v1:172.21.15.186:6789/0 conn(0x7fbcac0702f0 legacy=0x7fbcac070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb10d9640 1 -- 172.21.15.32:0/3835828627 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0ca7f0 con 0x7fbcac06a270 2023-10-16T08:17:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.506+0000 7fbcb08d8640 1 -- 172.21.15.32:0/3835828627 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323543800 0 0) 0x7fbcac15ae30 con 0x7fbcac0702f0 2023-10-16T08:17:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb10d9640 1 --2- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac06a270 0x7fbcac0c9ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009870 tx=0x7fbc9c01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ab7145dc86f1c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb08d8640 1 -- 172.21.15.32:0/3835828627 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0aa670 con 0x7fbcac06a270 2023-10-16T08:17:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb08d8640 1 -- 172.21.15.32:0/3835828627 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c0aa7d0 con 0x7fbcac06a270 2023-10-16T08:17:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb08d8640 1 -- 172.21.15.32:0/3835828627 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0aab60 con 0x7fbcac06a270 2023-10-16T08:17:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac06a270 msgr2=0x7fbcac0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac06a270 0x7fbcac0c9ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009870 tx=0x7fbc9c01c510 comp rx=0 tx=0).stop 2023-10-16T08:17:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/3835828627 shutdown_connections 2023-10-16T08:17:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac06a270 0x7fbcac0c9ce0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/3835828627 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac069e90 0x7fbcac06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/3835828627 >> 172.21.15.32:0/3835828627 conn(0x7fbcac0b0da0 msgr2=0x7fbcac0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:12.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/3835828627 shutdown_connections 2023-10-16T08:17:12.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/3835828627 wait complete. 2023-10-16T08:17:12.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 Processor -- start 2023-10-16T08:17:12.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- start start 2023-10-16T08:17:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac069e90 0x7fbcac1f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 0x7fbcac0aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac0702f0 0x7fbcac1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f6570 con 0x7fbcac06a270 2023-10-16T08:17:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f9a80 con 0x7fbcac069e90 2023-10-16T08:17:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.510+0000 7fbcb3b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f5fb0 con 0x7fbcac0702f0 2023-10-16T08:17:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 0x7fbcac0aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 0x7fbcac0aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39970/0 (socket says 172.21.15.32:39970) 2023-10-16T08:17:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 -- 172.21.15.32:0/1169142408 learned_addr learned my addr 172.21.15.32:0/1169142408 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb18da640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac069e90 0x7fbcac1f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb20db640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac0702f0 0x7fbcac1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 -- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac0702f0 msgr2=0x7fbcac1f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac0702f0 0x7fbcac1f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 -- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac069e90 msgr2=0x7fbcac1f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac069e90 0x7fbcac1f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 -- 172.21.15.32:0/1169142408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0094d0 con 0x7fbcac06a270 2023-10-16T08:17:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb10d9640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 0x7fbcac0aea10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0ac040 tx=0x7fbc9c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0aa670 con 0x7fbcac06a270 2023-10-16T08:17:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c0b3820 con 0x7fbcac06a270 2023-10-16T08:17:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0b3bd0 con 0x7fbcac06a270 2023-10-16T08:17:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac1f6290 con 0x7fbcac06a270 2023-10-16T08:17:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac1f9020 con 0x7fbcac06a270 2023-10-16T08:17:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac1f9210 con 0x7fbcac06a270 2023-10-16T08:17:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc9c0b3d30 con 0x7fbcac06a270 2023-10-16T08:17:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.514+0000 7fbca27fc640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbc900664a0 0x7fbc90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.518+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbc9c11ead0 con 0x7fbcac06a270 2023-10-16T08:17:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.518+0000 7fbcb18da640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbc900664a0 0x7fbc90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.518+0000 7fbcb18da640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbc900664a0 0x7fbc90068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009af0 tx=0x7fbca8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.518+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc9c0ebcd0 con 0x7fbcac06a270 2023-10-16T08:17:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcac04e170 con 0x7fbcac06a270 2023-10-16T08:17:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbca27fc640 1 -- 172.21.15.32:0/1169142408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc9c0c3070 con 0x7fbcac06a270 2023-10-16T08:17:12.684 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:12.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbc900664a0 msgr2=0x7fbc90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbc900664a0 0x7fbc90068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009af0 tx=0x7fbca8008040 comp rx=0 tx=0).stop 2023-10-16T08:17:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 msgr2=0x7fbcac0aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 0x7fbcac0aea10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0ac040 tx=0x7fbc9c01ad50 comp rx=0 tx=0).stop 2023-10-16T08:17:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 shutdown_connections 2023-10-16T08:17:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbc900664a0 0x7fbc90068960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbcac0702f0 0x7fbcac1f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcac06a270 0x7fbcac0aea10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 --2- 172.21.15.32:0/1169142408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcac069e90 0x7fbcac1f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 >> 172.21.15.32:0/1169142408 conn(0x7fbcac0b0da0 msgr2=0x7fbcac155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 shutdown_connections 2023-10-16T08:17:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.678+0000 7fbcb3b3e640 1 -- 172.21.15.32:0/1169142408 wait complete. 2023-10-16T08:17:12.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:12.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 Processor -- start 2023-10-16T08:17:12.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 -- start start 2023-10-16T08:17:12.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 0x7f55080d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f55080b84e0 0x7f55080cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55080d1670 con 0x7f55080d0040 2023-10-16T08:17:12.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55080d1460 con 0x7f55080b84e0 2023-10-16T08:17:12.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f550d7c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55080d04b0 con 0x7f550815cba0 2023-10-16T08:17:12.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 0x7f55080d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 0x7f55080d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:55582/0 (socket says 172.21.15.32:55582) 2023-10-16T08:17:12.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f54fffff640 1 --2- 172.21.15.32:0/729237073 >> v2:172.21.15.186:3300/0 conn(0x7f55080b84e0 0x7f55080cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 -- 172.21.15.32:0/729237073 learned_addr learned my addr 172.21.15.32:0/729237073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:12.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f55067fc640 1 -- 172.21.15.32:0/729237073 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380911729 0 0) 0x7f55080d1670 con 0x7f55080d0040 2023-10-16T08:17:12.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f55067fc640 1 -- 172.21.15.32:0/729237073 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f4003530 con 0x7f55080d0040 2023-10-16T08:17:12.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 -- 172.21.15.32:0/729237073 >> v2:172.21.15.186:3300/0 conn(0x7f55080b84e0 msgr2=0x7f55080cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:12.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 --2- 172.21.15.32:0/729237073 >> v2:172.21.15.186:3300/0 conn(0x7f55080b84e0 0x7f55080cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 -- 172.21.15.32:0/729237073 >> v1:172.21.15.186:6789/0 conn(0x7f55080d0040 legacy=0x7f55080d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.846+0000 7f5506ffd640 1 -- 172.21.15.32:0/729237073 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55080d1950 con 0x7f550815cba0 2023-10-16T08:17:12.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f5506ffd640 1 --2- 172.21.15.32:0/729237073 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 0x7f55080d0f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009870 tx=0x7f54f801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb561c545766cefd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f55067fc640 1 -- 172.21.15.32:0/729237073 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80ab660 con 0x7f550815cba0 2023-10-16T08:17:12.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f55067fc640 1 -- 172.21.15.32:0/729237073 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f80ab7c0 con 0x7f550815cba0 2023-10-16T08:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f55067fc640 1 -- 172.21.15.32:0/729237073 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80abb50 con 0x7f550815cba0 2023-10-16T08:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 -- 172.21.15.32:0/729237073 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 msgr2=0x7f55080d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 --2- 172.21.15.32:0/729237073 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 0x7f55080d0f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009870 tx=0x7f54f801c540 comp rx=0 tx=0).stop 2023-10-16T08:17:12.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 -- 172.21.15.32:0/729237073 shutdown_connections 2023-10-16T08:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 --2- 172.21.15.32:0/729237073 >> v2:172.21.15.186:3300/0 conn(0x7f55080b84e0 0x7f55080cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 --2- 172.21.15.32:0/729237073 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f550815cba0 0x7f55080d0f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 -- 172.21.15.32:0/729237073 >> 172.21.15.32:0/729237073 conn(0x7f55080719c0 msgr2=0x7f5508071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 -- 172.21.15.32:0/729237073 shutdown_connections 2023-10-16T08:17:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.850+0000 7f550d7c4640 1 -- 172.21.15.32:0/729237073 wait complete. 2023-10-16T08:17:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 Processor -- start 2023-10-16T08:17:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 -- start start 2023-10-16T08:17:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55080b84e0 0x7f5508147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55080d0040 0x7f5508150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 0x7f5508150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5508158f40 con 0x7f550815cba0 2023-10-16T08:17:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5508158330 con 0x7f55080b84e0 2023-10-16T08:17:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5508158610 con 0x7f55080d0040 2023-10-16T08:17:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 0x7f5508150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 0x7f5508150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42940/0 (socket says 172.21.15.32:42940) 2023-10-16T08:17:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f5506ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55080b84e0 0x7f5508147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 -- 172.21.15.32:0/4026042848 learned_addr learned my addr 172.21.15.32:0/4026042848 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f54fffff640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55080d0040 0x7f5508150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 -- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55080d0040 msgr2=0x7f5508150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55080d0040 0x7f5508150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 -- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55080b84e0 msgr2=0x7f5508147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:12.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55080b84e0 0x7f5508147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:12.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 -- 172.21.15.32:0/4026042848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80094d0 con 0x7f550815cba0 2023-10-16T08:17:12.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f55077fe640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 0x7f5508150e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f550000c790 tx=0x7f550000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000cea0 con 0x7f550815cba0 2023-10-16T08:17:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500003b80 con 0x7f550815cba0 2023-10-16T08:17:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55081588f0 con 0x7f550815cba0 2023-10-16T08:17:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000a850 con 0x7f550815cba0 2023-10-16T08:17:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.854+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5508148020 con 0x7f550815cba0 2023-10-16T08:17:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.858+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55081590f0 con 0x7f550815cba0 2023-10-16T08:17:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.858+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f550000a9b0 con 0x7f550815cba0 2023-10-16T08:17:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.858+0000 7f54ff7fe640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54e80664d0 0x7f54e8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.858+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5500083ff0 con 0x7f550815cba0 2023-10-16T08:17:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.858+0000 7f5506ffd640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54e80664d0 0x7f54e8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.858+0000 7f5506ffd640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54e80664d0 0x7f54e8068990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f54f80af000 tx=0x7f54f80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:12.862+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f55081590f0 con 0x7f550815cba0 2023-10-16T08:17:13.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.018+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f550804e170 con 0x7f550815cba0 2023-10-16T08:17:13.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.018+0000 7f54ff7fe640 1 -- 172.21.15.32:0/4026042848 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5500054b50 con 0x7f550815cba0 2023-10-16T08:17:13.026 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:13.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54e80664d0 msgr2=0x7f54e8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54e80664d0 0x7f54e8068990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f54f80af000 tx=0x7f54f80a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 msgr2=0x7f5508150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 0x7f5508150e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f550000c790 tx=0x7f550000cc50 comp rx=0 tx=0).stop 2023-10-16T08:17:13.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 shutdown_connections 2023-10-16T08:17:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f550815cba0 0x7f5508150e50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54e80664d0 0x7f54e8068990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f55080d0040 0x7f5508150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 --2- 172.21.15.32:0/4026042848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55080b84e0 0x7f5508147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 >> 172.21.15.32:0/4026042848 conn(0x7f55080719c0 msgr2=0x7f550815aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:13.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 shutdown_connections 2023-10-16T08:17:13.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.022+0000 7f550d7c4640 1 -- 172.21.15.32:0/4026042848 wait complete. 2023-10-16T08:17:13.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.186+0000 7f2d18f32640 1 Processor -- start 2023-10-16T08:17:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.186+0000 7f2d18f32640 1 -- start start 2023-10-16T08:17:13.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d18f32640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2d1415ba40 0x7f2d140ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d18f32640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1406d970 con 0x7f2d14154dc0 2023-10-16T08:17:13.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d18f32640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d140cf110 con 0x7f2d140cecd0 2023-10-16T08:17:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d18f32640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d141551a0 con 0x7f2d1415ba40 2023-10-16T08:17:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d137fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d14154dc0 0x7f2d1406d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45370/0 (socket says 172.21.15.32:45370) 2023-10-16T08:17:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d137fe640 1 -- 172.21.15.32:0/192279555 learned_addr learned my addr 172.21.15.32:0/192279555 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d12ffd640 1 --2- 172.21.15.32:0/192279555 >> v2:172.21.15.186:3300/0 conn(0x7f2d1415ba40 0x7f2d140ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d127fc640 1 -- 172.21.15.32:0/192279555 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220241548 0 0) 0x7f2d04001120 con 0x7f2d14154dc0 2023-10-16T08:17:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d127fc640 1 -- 172.21.15.32:0/192279555 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cfc003530 con 0x7f2d14154dc0 2023-10-16T08:17:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d12ffd640 1 -- 172.21.15.32:0/192279555 >> v1:172.21.15.186:6789/0 conn(0x7f2d140cecd0 legacy=0x7f2d140abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d12ffd640 1 -- 172.21.15.32:0/192279555 >> v1:172.21.15.32:6789/0 conn(0x7f2d14154dc0 legacy=0x7f2d1406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.190+0000 7f2d12ffd640 1 -- 172.21.15.32:0/192279555 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1406dbb0 con 0x7f2d1415ba40 2023-10-16T08:17:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d12ffd640 1 --2- 172.21.15.32:0/192279555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1415ba40 0x7f2d140ce790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d080097c0 tx=0x7f2d0801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f6f13cccd604eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d127fc640 1 -- 172.21.15.32:0/192279555 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d080ac620 con 0x7f2d1415ba40 2023-10-16T08:17:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d127fc640 1 -- 172.21.15.32:0/192279555 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d080ac780 con 0x7f2d1415ba40 2023-10-16T08:17:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d127fc640 1 -- 172.21.15.32:0/192279555 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d080acb10 con 0x7f2d1415ba40 2023-10-16T08:17:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 -- 172.21.15.32:0/192279555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1415ba40 msgr2=0x7f2d140ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 --2- 172.21.15.32:0/192279555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1415ba40 0x7f2d140ce790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d080097c0 tx=0x7f2d0801c3c0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 -- 172.21.15.32:0/192279555 shutdown_connections 2023-10-16T08:17:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 --2- 172.21.15.32:0/192279555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d1415ba40 0x7f2d140ce790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 -- 172.21.15.32:0/192279555 >> 172.21.15.32:0/192279555 conn(0x7f2d1406a290 msgr2=0x7f2d1406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 -- 172.21.15.32:0/192279555 shutdown_connections 2023-10-16T08:17:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 -- 172.21.15.32:0/192279555 wait complete. 2023-10-16T08:17:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 Processor -- start 2023-10-16T08:17:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.194+0000 7f2d18f32640 1 -- start start 2023-10-16T08:17:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d140cecd0 0x7f2d140979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14154dc0 0x7f2d1415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 0x7f2d140ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14097ee0 con 0x7f2d1415ba40 2023-10-16T08:17:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14071320 con 0x7f2d14154dc0 2023-10-16T08:17:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14070e90 con 0x7f2d140cecd0 2023-10-16T08:17:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 0x7f2d140ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 0x7f2d140ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42954/0 (socket says 172.21.15.32:42954) 2023-10-16T08:17:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 -- 172.21.15.32:0/3427559940 learned_addr learned my addr 172.21.15.32:0/3427559940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d137fe640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d140cecd0 0x7f2d140979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d12ffd640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14154dc0 0x7f2d1415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 -- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d140cecd0 msgr2=0x7f2d140979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d140cecd0 0x7f2d140979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 -- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14154dc0 msgr2=0x7f2d1415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14154dc0 0x7f2d1415a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 -- 172.21.15.32:0/3427559940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08009420 con 0x7f2d1415ba40 2023-10-16T08:17:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d13fff640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 0x7f2d140ccf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c720 tx=0x7f2d0c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c003b80 con 0x7f2d1415ba40 2023-10-16T08:17:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c003ce0 con 0x7f2d1415ba40 2023-10-16T08:17:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1415a5f0 con 0x7f2d1415ba40 2023-10-16T08:17:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0052f0 con 0x7f2d1415ba40 2023-10-16T08:17:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14070560 con 0x7f2d1415ba40 2023-10-16T08:17:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1404e9c0 con 0x7f2d1415ba40 2023-10-16T08:17:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.198+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2d0c005450 con 0x7f2d1415ba40 2023-10-16T08:17:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.202+0000 7f2cfbfff640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d000664d0 0x7f2d00068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.202+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2d0c083f70 con 0x7f2d1415ba40 2023-10-16T08:17:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.202+0000 7f2d137fe640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d000664d0 0x7f2d00068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.202+0000 7f2d137fe640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d000664d0 0x7f2d00068990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d04001280 tx=0x7f2d04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.202+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2d0c050f90 con 0x7f2d1415ba40 2023-10-16T08:17:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d1406cb30 con 0x7f2d1415ba40 2023-10-16T08:17:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2cfbfff640 1 -- 172.21.15.32:0/3427559940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d0c0545b0 con 0x7f2d1415ba40 2023-10-16T08:17:13.367 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d000664d0 msgr2=0x7f2d00068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d000664d0 0x7f2d00068990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d04001280 tx=0x7f2d04009040 comp rx=0 tx=0).stop 2023-10-16T08:17:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 msgr2=0x7f2d140ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 0x7f2d140ccf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c720 tx=0x7f2d0c00cbe0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 shutdown_connections 2023-10-16T08:17:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d1415ba40 0x7f2d140ccf90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14154dc0 0x7f2d1415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d000664d0 0x7f2d00068990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 --2- 172.21.15.32:0/3427559940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d140cecd0 0x7f2d140979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 >> 172.21.15.32:0/3427559940 conn(0x7f2d1406a290 msgr2=0x7f2d140addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 shutdown_connections 2023-10-16T08:17:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.362+0000 7f2d18f32640 1 -- 172.21.15.32:0/3427559940 wait complete. 2023-10-16T08:17:13.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:13.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.526+0000 7f548edc4640 1 Processor -- start 2023-10-16T08:17:13.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.526+0000 7f548edc4640 1 -- start start 2023-10-16T08:17:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548edc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548edc4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5488161390 con 0x7f5488160860 2023-10-16T08:17:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548edc4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54880ccda0 con 0x7f548806a5a0 2023-10-16T08:17:13.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548edc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880cd080 con 0x7f548815cbc0 2023-10-16T08:17:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f5487fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5488160860 0x7f5488069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45372/0 (socket says 172.21.15.32:45372) 2023-10-16T08:17:13.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f5487fff640 1 -- 172.21.15.32:0/3093332801 learned_addr learned my addr 172.21.15.32:0/3093332801 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548cb60640 1 --2- 172.21.15.32:0/3093332801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596800945 0 0) 0x7f5488161390 con 0x7f5488160860 2023-10-16T08:17:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5478003530 con 0x7f5488160860 2023-10-16T08:17:13.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449249444 0 0) 0x7f54880ccda0 con 0x7f548806a5a0 2023-10-16T08:17:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5488161390 con 0x7f548806a5a0 2023-10-16T08:17:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548cb60640 1 -- 172.21.15.32:0/3093332801 >> v1:172.21.15.186:6789/0 conn(0x7f548806a5a0 legacy=0x7f54880cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548cb60640 1 -- 172.21.15.32:0/3093332801 >> v1:172.21.15.32:6789/0 conn(0x7f5488160860 legacy=0x7f5488069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186989754 0 0) 0x7f5478003530 con 0x7f5488160860 2023-10-16T08:17:13.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.530+0000 7f548cb60640 1 -- 172.21.15.32:0/3093332801 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880cd360 con 0x7f548815cbc0 2023-10-16T08:17:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548cb60640 1 --2- 172.21.15.32:0/3093332801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880ab870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f547c0097f0 tx=0x7f547c01c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b30dfad9ea8a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474001680 con 0x7f548815cbc0 2023-10-16T08:17:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c0a7050 con 0x7f548815cbc0 2023-10-16T08:17:13.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f54877fe640 1 -- 172.21.15.32:0/3093332801 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0ab620 con 0x7f548815cbc0 2023-10-16T08:17:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 -- 172.21.15.32:0/3093332801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 msgr2=0x7f54880ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 --2- 172.21.15.32:0/3093332801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880ab870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f547c0097f0 tx=0x7f547c01c350 comp rx=0 tx=0).stop 2023-10-16T08:17:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 -- 172.21.15.32:0/3093332801 shutdown_connections 2023-10-16T08:17:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 --2- 172.21.15.32:0/3093332801 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880ab870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 -- 172.21.15.32:0/3093332801 >> 172.21.15.32:0/3093332801 conn(0x7f54880b0da0 msgr2=0x7f54880ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 -- 172.21.15.32:0/3093332801 shutdown_connections 2023-10-16T08:17:13.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 -- 172.21.15.32:0/3093332801 wait complete. 2023-10-16T08:17:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 Processor -- start 2023-10-16T08:17:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.534+0000 7f548edc4640 1 -- start start 2023-10-16T08:17:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548806a5a0 0x7f548813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 0x7f54880cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548815deb0 con 0x7f5488160860 2023-10-16T08:17:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54881420c0 con 0x7f548806a5a0 2023-10-16T08:17:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5488142310 con 0x7f548815cbc0 2023-10-16T08:17:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 0x7f54880cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 0x7f54880cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42970/0 (socket says 172.21.15.32:42970) 2023-10-16T08:17:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 -- 172.21.15.32:0/906216137 learned_addr learned my addr 172.21.15.32:0/906216137 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548cb60640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548806a5a0 0x7f548813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f5487fff640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 -- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 msgr2=0x7f54880cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 -- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548806a5a0 msgr2=0x7f548813f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548806a5a0 0x7f548813f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 -- 172.21.15.32:0/906216137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c009420 con 0x7f5488160860 2023-10-16T08:17:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f5487fff640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548d361640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 0x7f54880cb690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f548000c7a0 tx=0x7f548000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480003b80 con 0x7f5488160860 2023-10-16T08:17:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480003ce0 con 0x7f5488160860 2023-10-16T08:17:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000aa90 con 0x7f5488160860 2023-10-16T08:17:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54881425f0 con 0x7f5488160860 2023-10-16T08:17:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5488142bb0 con 0x7f5488160860 2023-10-16T08:17:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.538+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488142da0 con 0x7f5488160860 2023-10-16T08:17:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.542+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f548000abf0 con 0x7f5488160860 2023-10-16T08:17:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.542+0000 7f54857fa640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f546c0664a0 0x7f546c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.542+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5480084050 con 0x7f5488160860 2023-10-16T08:17:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.542+0000 7f548cb60640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f546c0664a0 0x7f546c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.542+0000 7f548cb60640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f546c0664a0 0x7f546c068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f547c0ad000 tx=0x7f547c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.546+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5480051110 con 0x7f5488160860 2023-10-16T08:17:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.698+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f548804e170 con 0x7f5488160860 2023-10-16T08:17:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.698+0000 7f54857fa640 1 -- 172.21.15.32:0/906216137 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5480054730 con 0x7f5488160860 2023-10-16T08:17:13.706 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f546c0664a0 msgr2=0x7f546c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f546c0664a0 0x7f546c068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f547c0ad000 tx=0x7f547c0a6000 comp rx=0 tx=0).stop 2023-10-16T08:17:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 msgr2=0x7f54880cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 0x7f54880cb690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f548000c7a0 tx=0x7f548000cc60 comp rx=0 tx=0).stop 2023-10-16T08:17:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 shutdown_connections 2023-10-16T08:17:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5488160860 0x7f54880cb690 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f546c0664a0 0x7f546c068960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f548815cbc0 0x7f54880cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 --2- 172.21.15.32:0/906216137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f548806a5a0 0x7f548813f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 >> 172.21.15.32:0/906216137 conn(0x7f54880b0da0 msgr2=0x7f5488069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:13.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 shutdown_connections 2023-10-16T08:17:13.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.702+0000 7f548edc4640 1 -- 172.21.15.32:0/906216137 wait complete. 2023-10-16T08:17:13.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 Processor -- start 2023-10-16T08:17:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 -- start start 2023-10-16T08:17:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3c7815a880 0x7f3c7815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c780cda30 0x7f3c780b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7815e000 con 0x7f3c780b00e0 2023-10-16T08:17:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780b0ab0 con 0x7f3c780cda30 2023-10-16T08:17:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7eb65640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780cb1d0 con 0x7f3c7815a880 2023-10-16T08:17:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c780cda30 0x7f3c780b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c780cda30 0x7f3c780b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42974/0 (socket says 172.21.15.32:42974) 2023-10-16T08:17:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 -- 172.21.15.32:0/363562629 learned_addr learned my addr 172.21.15.32:0/363562629 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c77fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c780b00e0 0x7f3c78160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45374/0 (socket says 172.21.15.32:45374) 2023-10-16T08:17:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7c901640 1 --2- 172.21.15.32:0/363562629 >> v2:172.21.15.186:3300/0 conn(0x7f3c7815a880 0x7f3c7815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 -- 172.21.15.32:0/363562629 >> v2:172.21.15.186:3300/0 conn(0x7f3c7815a880 msgr2=0x7f3c7815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 --2- 172.21.15.32:0/363562629 >> v2:172.21.15.186:3300/0 conn(0x7f3c7815a880 0x7f3c7815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 -- 172.21.15.32:0/363562629 >> v1:172.21.15.32:6789/0 conn(0x7f3c780b00e0 legacy=0x7f3c78160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7d102640 1 -- 172.21.15.32:0/363562629 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780cb4b0 con 0x7f3c780cda30 2023-10-16T08:17:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.866+0000 7f3c7c901640 1 --2- 172.21.15.32:0/363562629 >> v2:172.21.15.186:3300/0 conn(0x7f3c7815a880 0x7f3c7815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7d102640 1 --2- 172.21.15.32:0/363562629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780cda30 0x7f3c780b0570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c7806ae80 tx=0x7f3c7001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb6b6588c21c740 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c777fe640 1 -- 172.21.15.32:0/363562629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700a56a0 con 0x7f3c780cda30 2023-10-16T08:17:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c777fe640 1 -- 172.21.15.32:0/363562629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c700a5800 con 0x7f3c780cda30 2023-10-16T08:17:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c777fe640 1 -- 172.21.15.32:0/363562629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700a5b90 con 0x7f3c780cda30 2023-10-16T08:17:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 -- 172.21.15.32:0/363562629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780cda30 msgr2=0x7f3c780b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/363562629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780cda30 0x7f3c780b0570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c7806ae80 tx=0x7f3c7001c9d0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 -- 172.21.15.32:0/363562629 shutdown_connections 2023-10-16T08:17:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/363562629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780cda30 0x7f3c780b0570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/363562629 >> v2:172.21.15.186:3300/0 conn(0x7f3c7815a880 0x7f3c7815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 -- 172.21.15.32:0/363562629 >> 172.21.15.32:0/363562629 conn(0x7f3c7806c190 msgr2=0x7f3c7806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 -- 172.21.15.32:0/363562629 shutdown_connections 2023-10-16T08:17:13.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.870+0000 7f3c7eb65640 1 -- 172.21.15.32:0/363562629 wait complete. 2023-10-16T08:17:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 Processor -- start 2023-10-16T08:17:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- start start 2023-10-16T08:17:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 0x7f3c7815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c780cda30 0x7f3c78152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3c7815a880 0x7f3c780ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c78154490 con 0x7f3c780b00e0 2023-10-16T08:17:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780ac310 con 0x7f3c780cda30 2023-10-16T08:17:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780ac5f0 con 0x7f3c7815a880 2023-10-16T08:17:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 0x7f3c7815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 0x7f3c7815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42984/0 (socket says 172.21.15.32:42984) 2023-10-16T08:17:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 -- 172.21.15.32:0/3485731964 learned_addr learned my addr 172.21.15.32:0/3485731964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c77fff640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c780cda30 0x7f3c78152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7d102640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3c7815a880 0x7f3c780ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 -- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3c7815a880 msgr2=0x7f3c780ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3c7815a880 0x7f3c780ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 -- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c780cda30 msgr2=0x7f3c78152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c780cda30 0x7f3c78152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 -- 172.21.15.32:0/3485731964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c70009b30 con 0x7f3c780b00e0 2023-10-16T08:17:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7d102640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3c7815a880 0x7f3c780ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7c901640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 0x7f3c7815b9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c7c0 tx=0x7f3c6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c007a30 con 0x7f3c780b00e0 2023-10-16T08:17:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c007b90 con 0x7f3c780b00e0 2023-10-16T08:17:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c780ac8d0 con 0x7f3c780b00e0 2023-10-16T08:17:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00ce70 con 0x7f3c780b00e0 2023-10-16T08:17:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c78152ed0 con 0x7f3c780b00e0 2023-10-16T08:17:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.874+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c780cad80 con 0x7f3c780b00e0 2023-10-16T08:17:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.878+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c6c0113f0 con 0x7f3c780b00e0 2023-10-16T08:17:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.878+0000 7f3c757fa640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c5c0664a0 0x7f3c5c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.878+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3c6c084720 con 0x7f3c780b00e0 2023-10-16T08:17:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.878+0000 7f3c77fff640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c5c0664a0 0x7f3c5c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.878+0000 7f3c77fff640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c5c0664a0 0x7f3c5c068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c7815af20 tx=0x7f3c64007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:13.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:13.882+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3c6c051750 con 0x7f3c780b00e0 2023-10-16T08:17:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c780acd80 con 0x7f3c780b00e0 2023-10-16T08:17:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c757fa640 1 -- 172.21.15.32:0/3485731964 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f3c6c054d70 con 0x7f3c780b00e0 2023-10-16T08:17:14.047 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:14.048 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c5c0664a0 msgr2=0x7f3c5c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c5c0664a0 0x7f3c5c068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c7815af20 tx=0x7f3c64007c90 comp rx=0 tx=0).stop 2023-10-16T08:17:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 msgr2=0x7f3c7815b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 0x7f3c7815b9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c7c0 tx=0x7f3c6c00cc80 comp rx=0 tx=0).stop 2023-10-16T08:17:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 shutdown_connections 2023-10-16T08:17:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3c5c0664a0 0x7f3c5c068960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3c7815a880 0x7f3c780ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c780cda30 0x7f3c78152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 --2- 172.21.15.32:0/3485731964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c780b00e0 0x7f3c7815b9d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 >> 172.21.15.32:0/3485731964 conn(0x7f3c7806c190 msgr2=0x7f3c78153610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 shutdown_connections 2023-10-16T08:17:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.042+0000 7f3c7eb65640 1 -- 172.21.15.32:0/3485731964 wait complete. 2023-10-16T08:17:14.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:14.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.210+0000 7f8142609640 1 Processor -- start 2023-10-16T08:17:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.210+0000 7f8142609640 1 -- start start 2023-10-16T08:17:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8142609640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f813c06a5a0 0x7f813c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8142609640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c0b3f00 0x7f813c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8142609640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c0b4370 con 0x7f813c159900 2023-10-16T08:17:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8142609640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c0cd740 con 0x7f813c06a5a0 2023-10-16T08:17:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8142609640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0b4600 con 0x7f813c0b3f00 2023-10-16T08:17:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f813c159900 0x7f813c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45386/0 (socket says 172.21.15.32:45386) 2023-10-16T08:17:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813bfff640 1 -- 172.21.15.32:0/1513322362 learned_addr learned my addr 172.21.15.32:0/1513322362 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813b7fe640 1 --2- 172.21.15.32:0/1513322362 >> v2:172.21.15.186:3300/0 conn(0x7f813c06a5a0 0x7f813c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8140ba6640 1 --2- 172.21.15.32:0/1513322362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c0b3f00 0x7f813c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813affd640 1 -- 172.21.15.32:0/1513322362 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314190060 0 0) 0x7f813c0b4370 con 0x7f813c159900 2023-10-16T08:17:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813affd640 1 -- 172.21.15.32:0/1513322362 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8124005740 con 0x7f813c159900 2023-10-16T08:17:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813affd640 1 -- 172.21.15.32:0/1513322362 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2276491596 0 0) 0x7f8124005740 con 0x7f813c159900 2023-10-16T08:17:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8140ba6640 1 -- 172.21.15.32:0/1513322362 >> v2:172.21.15.186:3300/0 conn(0x7f813c06a5a0 msgr2=0x7f813c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8140ba6640 1 --2- 172.21.15.32:0/1513322362 >> v2:172.21.15.186:3300/0 conn(0x7f813c06a5a0 0x7f813c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8140ba6640 1 -- 172.21.15.32:0/1513322362 >> v1:172.21.15.32:6789/0 conn(0x7f813c159900 legacy=0x7f813c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f8140ba6640 1 -- 172.21.15.32:0/1513322362 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0c8dd0 con 0x7f813c0b3f00 2023-10-16T08:17:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.214+0000 7f813b7fe640 1 --2- 172.21.15.32:0/1513322362 >> v2:172.21.15.186:3300/0 conn(0x7f813c06a5a0 0x7f813c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8140ba6640 1 --2- 172.21.15.32:0/1513322362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c0b3f00 0x7f813c0c9b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8134009d90 tx=0x7f813401c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e66716715aa718 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f813affd640 1 -- 172.21.15.32:0/1513322362 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c001680 con 0x7f813c0b3f00 2023-10-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f813affd640 1 -- 172.21.15.32:0/1513322362 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81340a5640 con 0x7f813c0b3f00 2023-10-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f813affd640 1 -- 172.21.15.32:0/1513322362 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81340a59d0 con 0x7f813c0b3f00 2023-10-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 -- 172.21.15.32:0/1513322362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c0b3f00 msgr2=0x7f813c0c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 --2- 172.21.15.32:0/1513322362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c0b3f00 0x7f813c0c9b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8134009d90 tx=0x7f813401c840 comp rx=0 tx=0).stop 2023-10-16T08:17:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 -- 172.21.15.32:0/1513322362 shutdown_connections 2023-10-16T08:17:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 --2- 172.21.15.32:0/1513322362 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c0b3f00 0x7f813c0c9b90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 --2- 172.21.15.32:0/1513322362 >> v2:172.21.15.186:3300/0 conn(0x7f813c06a5a0 0x7f813c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 -- 172.21.15.32:0/1513322362 >> 172.21.15.32:0/1513322362 conn(0x7f813c0b0da0 msgr2=0x7f813c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 -- 172.21.15.32:0/1513322362 shutdown_connections 2023-10-16T08:17:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 -- 172.21.15.32:0/1513322362 wait complete. 2023-10-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 Processor -- start 2023-10-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.218+0000 7f8142609640 1 -- start start 2023-10-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c06a5a0 0x7f813c0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 0x7f813c1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c159900 0x7f813c1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0c7d80 con 0x7f813c0b3f00 2023-10-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c15b710 con 0x7f813c06a5a0 2023-10-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0c7f60 con 0x7f813c159900 2023-10-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 0x7f813c1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 0x7f813c1f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42996/0 (socket says 172.21.15.32:42996) 2023-10-16T08:17:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 -- 172.21.15.32:0/3033853376 learned_addr learned my addr 172.21.15.32:0/3033853376 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813bfff640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c06a5a0 0x7f813c0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8140ba6640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c159900 0x7f813c1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 -- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c159900 msgr2=0x7f813c1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c159900 0x7f813c1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 -- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c06a5a0 msgr2=0x7f813c0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c06a5a0 0x7f813c0c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 -- 172.21.15.32:0/3033853376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130009420 con 0x7f813c0b3f00 2023-10-16T08:17:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813bfff640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c06a5a0 0x7f813c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8140ba6640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c159900 0x7f813c1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f813b7fe640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 0x7f813c1f4bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f813000ebe0 tx=0x7f813000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000eee0 con 0x7f813c0b3f00 2023-10-16T08:17:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000cc30 con 0x7f813c0b3f00 2023-10-16T08:17:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8134009a70 con 0x7f813c0b3f00 2023-10-16T08:17:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130013470 con 0x7f813c0b3f00 2023-10-16T08:17:14.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c0c8520 con 0x7f813c0b3f00 2023-10-16T08:17:14.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.222+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c04e170 con 0x7f813c0b3f00 2023-10-16T08:17:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.226+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81300135d0 con 0x7f813c0b3f00 2023-10-16T08:17:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.226+0000 7f8138ff9640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81280664a0 0x7f8128068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.226+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f81300845e0 con 0x7f813c0b3f00 2023-10-16T08:17:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.226+0000 7f813bfff640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81280664a0 0x7f8128068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.226+0000 7f813bfff640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81280664a0 0x7f8128068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f812c005890 tx=0x7f812c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.230+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8130051600 con 0x7f813c0b3f00 2023-10-16T08:17:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f813c073fa0 con 0x7f813c0b3f00 2023-10-16T08:17:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8138ff9640 1 -- 172.21.15.32:0/3033853376 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8130054c20 con 0x7f813c0b3f00 2023-10-16T08:17:14.392 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81280664a0 msgr2=0x7f8128068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81280664a0 0x7f8128068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f812c005890 tx=0x7f812c009040 comp rx=0 tx=0).stop 2023-10-16T08:17:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 msgr2=0x7f813c1f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 0x7f813c1f4bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f813000ebe0 tx=0x7f813000c520 comp rx=0 tx=0).stop 2023-10-16T08:17:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 shutdown_connections 2023-10-16T08:17:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81280664a0 0x7f8128068960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f813c159900 0x7f813c1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f813c0b3f00 0x7f813c1f4bb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 --2- 172.21.15.32:0/3033853376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f813c06a5a0 0x7f813c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 >> 172.21.15.32:0/3033853376 conn(0x7f813c0b0da0 msgr2=0x7f813c072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 shutdown_connections 2023-10-16T08:17:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.386+0000 7f8142609640 1 -- 172.21.15.32:0/3033853376 wait complete. 2023-10-16T08:17:14.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964c30f640 1 Processor -- start 2023-10-16T08:17:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964c30f640 1 -- start start 2023-10-16T08:17:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964c30f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cb040 0x7f96440ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964c30f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440cc330 con 0x7f964415cc60 2023-10-16T08:17:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964c30f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440cb4a0 con 0x7f96440cebd0 2023-10-16T08:17:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964c30f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440cefe0 con 0x7f96440cb040 2023-10-16T08:17:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964a0ab640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f964415cc60 0x7f9644161260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45392/0 (socket says 172.21.15.32:45392) 2023-10-16T08:17:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.554+0000 7f964a0ab640 1 -- 172.21.15.32:0/886826397 learned_addr learned my addr 172.21.15.32:0/886826397 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96498aa640 1 --2- 172.21.15.32:0/886826397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cb040 0x7f96440ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145389154 0 0) 0x7f96440cc330 con 0x7f964415cc60 2023-10-16T08:17:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9630003530 con 0x7f964415cc60 2023-10-16T08:17:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263242521 0 0) 0x7f96440cb4a0 con 0x7f96440cebd0 2023-10-16T08:17:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96440cc330 con 0x7f96440cebd0 2023-10-16T08:17:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96498aa640 1 -- 172.21.15.32:0/886826397 >> v1:172.21.15.186:6789/0 conn(0x7f96440cebd0 legacy=0x7f96440cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2345656658 0 0) 0x7f9630003530 con 0x7f964415cc60 2023-10-16T08:17:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96498aa640 1 -- 172.21.15.32:0/886826397 >> v1:172.21.15.32:6789/0 conn(0x7f964415cc60 legacy=0x7f9644161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96498aa640 1 -- 172.21.15.32:0/886826397 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440cb720 con 0x7f96440cb040 2023-10-16T08:17:14.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96498aa640 1 --2- 172.21.15.32:0/886826397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cb040 0x7f96440ce690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96340097f0 tx=0x7f963401c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7590e87322d9136 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640001680 con 0x7f96440cb040 2023-10-16T08:17:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96340a7050 con 0x7f96440cb040 2023-10-16T08:17:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.558+0000 7f96490a9640 1 -- 172.21.15.32:0/886826397 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340ab620 con 0x7f96440cb040 2023-10-16T08:17:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- 172.21.15.32:0/886826397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cb040 msgr2=0x7f96440ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 --2- 172.21.15.32:0/886826397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cb040 0x7f96440ce690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96340097f0 tx=0x7f963401c3b0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- 172.21.15.32:0/886826397 shutdown_connections 2023-10-16T08:17:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 --2- 172.21.15.32:0/886826397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cb040 0x7f96440ce690 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- 172.21.15.32:0/886826397 >> 172.21.15.32:0/886826397 conn(0x7f964406e570 msgr2=0x7f964406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- 172.21.15.32:0/886826397 shutdown_connections 2023-10-16T08:17:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- 172.21.15.32:0/886826397 wait complete. 2023-10-16T08:17:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 Processor -- start 2023-10-16T08:17:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- start start 2023-10-16T08:17:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 0x7f9644151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cebd0 0x7f96440c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f964415cc60 0x7f964415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9644152200 con 0x7f96440cb040 2023-10-16T08:17:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c5e40 con 0x7f964415cc60 2023-10-16T08:17:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964c30f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c60f0 con 0x7f96440cebd0 2023-10-16T08:17:14.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 0x7f9644151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 0x7f9644151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43006/0 (socket says 172.21.15.32:43006) 2023-10-16T08:17:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 -- 172.21.15.32:0/1938347777 learned_addr learned my addr 172.21.15.32:0/1938347777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f96498aa640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cebd0 0x7f96440c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a8ac640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f964415cc60 0x7f964415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 -- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cebd0 msgr2=0x7f96440c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cebd0 0x7f96440c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 -- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f964415cc60 msgr2=0x7f964415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f964415cc60 0x7f964415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a0ab640 1 -- 172.21.15.32:0/1938347777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634009420 con 0x7f96440cb040 2023-10-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.562+0000 7f964a8ac640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f964415cc60 0x7f964415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:14.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f964a0ab640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 0x7f9644151cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f964000c750 tx=0x7f964000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640005ae0 con 0x7f96440cb040 2023-10-16T08:17:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640005c40 con 0x7f96440cb040 2023-10-16T08:17:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96440c6be0 con 0x7f96440cb040 2023-10-16T08:17:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640007070 con 0x7f96440cb040 2023-10-16T08:17:14.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96440c6620 con 0x7f96440cb040 2023-10-16T08:17:14.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964404e170 con 0x7f96440cb040 2023-10-16T08:17:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96400071d0 con 0x7f96440cb040 2023-10-16T08:17:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f963affd640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96280664d0 0x7f9628068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9640083e80 con 0x7f96440cb040 2023-10-16T08:17:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.566+0000 7f96498aa640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96280664d0 0x7f9628068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.570+0000 7f96498aa640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96280664d0 0x7f9628068990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96340ad000 tx=0x7f9634008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.570+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9640050f30 con 0x7f96440cb040 2023-10-16T08:17:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96440b6830 con 0x7f96440cb040 2023-10-16T08:17:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f963affd640 1 -- 172.21.15.32:0/1938347777 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9640054550 con 0x7f96440cb040 2023-10-16T08:17:14.733 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:14.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96280664d0 msgr2=0x7f9628068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96280664d0 0x7f9628068990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96340ad000 tx=0x7f9634008040 comp rx=0 tx=0).stop 2023-10-16T08:17:14.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 msgr2=0x7f9644151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 0x7f9644151cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f964000c750 tx=0x7f964000cc10 comp rx=0 tx=0).stop 2023-10-16T08:17:14.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 shutdown_connections 2023-10-16T08:17:14.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f964415cc60 0x7f964415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96280664d0 0x7f9628068990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96440cebd0 0x7f96440c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 --2- 172.21.15.32:0/1938347777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96440cb040 0x7f9644151cc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.726+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 >> 172.21.15.32:0/1938347777 conn(0x7f964406e570 msgr2=0x7f964415f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.730+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 shutdown_connections 2023-10-16T08:17:14.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.730+0000 7f964c30f640 1 -- 172.21.15.32:0/1938347777 wait complete. 2023-10-16T08:17:14.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.894+0000 7f2c3c9f1640 1 Processor -- start 2023-10-16T08:17:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.894+0000 7f2c3c9f1640 1 -- start start 2023-10-16T08:17:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3c9f1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2c3815cba0 0x7f2c380d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3c9f1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 0x7f2c380cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3c9f1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c380d1670 con 0x7f2c380d0040 2023-10-16T08:17:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3c9f1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c380d1460 con 0x7f2c3815cba0 2023-10-16T08:17:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3c9f1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380d04b0 con 0x7f2c380b84e0 2023-10-16T08:17:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 0x7f2c380cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2c3815cba0 0x7f2c380d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 0x7f2c380cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:55678/0 (socket says 172.21.15.32:55678) 2023-10-16T08:17:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 -- 172.21.15.32:0/1545056264 learned_addr learned my addr 172.21.15.32:0/1545056264 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 -- 172.21.15.32:0/1545056264 >> v2:172.21.15.186:3300/0 conn(0x7f2c3815cba0 msgr2=0x7f2c380d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 --2- 172.21.15.32:0/1545056264 >> v2:172.21.15.186:3300/0 conn(0x7f2c3815cba0 0x7f2c380d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 -- 172.21.15.32:0/1545056264 >> v1:172.21.15.186:6789/0 conn(0x7f2c380d0040 legacy=0x7f2c380d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c35d9b640 1 -- 172.21.15.32:0/1545056264 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380d1950 con 0x7f2c380b84e0 2023-10-16T08:17:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.898+0000 7f2c3659c640 1 --2- 172.21.15.32:0/1545056264 >> v2:172.21.15.186:3300/0 conn(0x7f2c3815cba0 0x7f2c380d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c35d9b640 1 --2- 172.21.15.32:0/1545056264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 0x7f2c380cfb00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009870 tx=0x7f2c2c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c690b662732c7c24 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3559a640 1 -- 172.21.15.32:0/1545056264 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0ab660 con 0x7f2c380b84e0 2023-10-16T08:17:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3559a640 1 -- 172.21.15.32:0/1545056264 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0ab7c0 con 0x7f2c380b84e0 2023-10-16T08:17:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3559a640 1 -- 172.21.15.32:0/1545056264 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0abb50 con 0x7f2c380b84e0 2023-10-16T08:17:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/1545056264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 msgr2=0x7f2c380cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/1545056264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 0x7f2c380cfb00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009870 tx=0x7f2c2c01c540 comp rx=0 tx=0).stop 2023-10-16T08:17:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/1545056264 shutdown_connections 2023-10-16T08:17:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/1545056264 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380b84e0 0x7f2c380cfb00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/1545056264 >> v2:172.21.15.186:3300/0 conn(0x7f2c3815cba0 0x7f2c380d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/1545056264 >> 172.21.15.32:0/1545056264 conn(0x7f2c380719c0 msgr2=0x7f2c38071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/1545056264 shutdown_connections 2023-10-16T08:17:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.902+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/1545056264 wait complete. 2023-10-16T08:17:14.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 Processor -- start 2023-10-16T08:17:14.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 -- start start 2023-10-16T08:17:14.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c380b84e0 0x7f2c380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380d0040 0x7f2c380cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c3815cba0 0x7f2c3815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380cbc20 con 0x7f2c3815cba0 2023-10-16T08:17:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380cc5f0 con 0x7f2c380b84e0 2023-10-16T08:17:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380cb010 con 0x7f2c380d0040 2023-10-16T08:17:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c3815cba0 0x7f2c3815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c35d9b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380d0040 0x7f2c380cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c380b84e0 0x7f2c380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c380b84e0 0x7f2c380d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:47330/0 (socket says 172.21.15.32:47330) 2023-10-16T08:17:14.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3659c640 1 -- 172.21.15.32:0/3416001824 learned_addr learned my addr 172.21.15.32:0/3416001824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:14.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 -- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380d0040 msgr2=0x7f2c380cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380d0040 0x7f2c380cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 -- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c380b84e0 msgr2=0x7f2c380d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:14.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c380b84e0 0x7f2c380d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:14.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 -- 172.21.15.32:0/3416001824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c28009420 con 0x7f2c3815cba0 2023-10-16T08:17:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c35d9b640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380d0040 0x7f2c380cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c36d9d640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c3815cba0 0x7f2c3815ebf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c700 tx=0x7f2c3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30003b80 con 0x7f2c3815cba0 2023-10-16T08:17:14.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30003ce0 con 0x7f2c3815cba0 2023-10-16T08:17:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000aa30 con 0x7f2c3815cba0 2023-10-16T08:17:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c0094d0 con 0x7f2c3815cba0 2023-10-16T08:17:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.906+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c380cb5d0 con 0x7f2c3815cba0 2023-10-16T08:17:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.910+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c380cbd90 con 0x7f2c3815cba0 2023-10-16T08:17:14.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.910+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2c3000ab90 con 0x7f2c3815cba0 2023-10-16T08:17:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.910+0000 7f2c26ffd640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2c180664a0 0x7f2c18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:14.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.910+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2c30083ec0 con 0x7f2c3815cba0 2023-10-16T08:17:14.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.910+0000 7f2c3659c640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2c180664a0 0x7f2c18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:14.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.910+0000 7f2c3659c640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2c180664a0 0x7f2c18068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009d50 tx=0x7f2c28009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:14.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:14.914+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2c30051030 con 0x7f2c3815cba0 2023-10-16T08:17:15.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c3804e170 con 0x7f2c3815cba0 2023-10-16T08:17:15.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c26ffd640 1 -- 172.21.15.32:0/3416001824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c30054650 con 0x7f2c3815cba0 2023-10-16T08:17:15.079 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:15.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2c180664a0 msgr2=0x7f2c18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2c180664a0 0x7f2c18068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009d50 tx=0x7f2c28009190 comp rx=0 tx=0).stop 2023-10-16T08:17:15.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c3815cba0 msgr2=0x7f2c3815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c3815cba0 0x7f2c3815ebf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c700 tx=0x7f2c3000cbc0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 shutdown_connections 2023-10-16T08:17:15.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c3815cba0 0x7f2c3815ebf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2c180664a0 0x7f2c18068960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c380d0040 0x7f2c380cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 --2- 172.21.15.32:0/3416001824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c380b84e0 0x7f2c380d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 >> 172.21.15.32:0/3416001824 conn(0x7f2c380719c0 msgr2=0x7f2c380b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:15.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 shutdown_connections 2023-10-16T08:17:15.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.074+0000 7f2c3c9f1640 1 -- 172.21.15.32:0/3416001824 wait complete. 2023-10-16T08:17:15.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.242+0000 7f1293565640 1 Processor -- start 2023-10-16T08:17:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.242+0000 7f1293565640 1 -- start start 2023-10-16T08:17:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1293565640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f128c06d010 0x7f128c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1293565640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c15ae30 con 0x7f128c0c26b0 2023-10-16T08:17:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1293565640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c0ca510 con 0x7f128c069e90 2023-10-16T08:17:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1293565640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c0cb090 con 0x7f128c06d010 2023-10-16T08:17:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1291b02640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f128c0c26b0 0x7f128c0c2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45396/0 (socket says 172.21.15.32:45396) 2023-10-16T08:17:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1291b02640 1 -- 172.21.15.32:0/1632312865 learned_addr learned my addr 172.21.15.32:0/1632312865 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1290b00640 1 --2- 172.21.15.32:0/1632312865 >> v2:172.21.15.186:3300/0 conn(0x7f128c06d010 0x7f128c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476087210 0 0) 0x7f128c15ae30 con 0x7f128c0c26b0 2023-10-16T08:17:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1278003530 con 0x7f128c0c26b0 2023-10-16T08:17:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483928103 0 0) 0x7f128c0ca510 con 0x7f128c069e90 2023-10-16T08:17:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f128c15ae30 con 0x7f128c069e90 2023-10-16T08:17:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1290b00640 1 -- 172.21.15.32:0/1632312865 >> v1:172.21.15.186:6789/0 conn(0x7f128c069e90 legacy=0x7f128c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1290b00640 1 -- 172.21.15.32:0/1632312865 >> v1:172.21.15.32:6789/0 conn(0x7f128c0c26b0 legacy=0x7f128c0c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.246+0000 7f1290b00640 1 -- 172.21.15.32:0/1632312865 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0ca7f0 con 0x7f128c06d010 2023-10-16T08:17:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1290b00640 1 --2- 172.21.15.32:0/1632312865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c0c9ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097c0 tx=0x7f127c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc1a05543047b71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0a8970 con 0x7f128c06d010 2023-10-16T08:17:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c0a8ad0 con 0x7f128c06d010 2023-10-16T08:17:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1283fff640 1 -- 172.21.15.32:0/1632312865 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0a8e60 con 0x7f128c06d010 2023-10-16T08:17:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- 172.21.15.32:0/1632312865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 msgr2=0x7f128c0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 --2- 172.21.15.32:0/1632312865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c0c9ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097c0 tx=0x7f127c01ec70 comp rx=0 tx=0).stop 2023-10-16T08:17:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- 172.21.15.32:0/1632312865 shutdown_connections 2023-10-16T08:17:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 --2- 172.21.15.32:0/1632312865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c0c9ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- 172.21.15.32:0/1632312865 >> 172.21.15.32:0/1632312865 conn(0x7f128c0b0da0 msgr2=0x7f128c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- 172.21.15.32:0/1632312865 shutdown_connections 2023-10-16T08:17:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- 172.21.15.32:0/1632312865 wait complete. 2023-10-16T08:17:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 Processor -- start 2023-10-16T08:17:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- start start 2023-10-16T08:17:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f128c069e90 0x7f128c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c1e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 0x7f128c1e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c15dbd0 con 0x7f128c0c26b0 2023-10-16T08:17:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c15d8f0 con 0x7f128c06d010 2023-10-16T08:17:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.250+0000 7f1293565640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0c6880 con 0x7f128c069e90 2023-10-16T08:17:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 0x7f128c1e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 0x7f128c1e3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43030/0 (socket says 172.21.15.32:43030) 2023-10-16T08:17:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 -- 172.21.15.32:0/1509569915 learned_addr learned my addr 172.21.15.32:0/1509569915 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1290b00640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c1e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291301640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f128c069e90 0x7f128c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 -- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f128c069e90 msgr2=0x7f128c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f128c069e90 0x7f128c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 -- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 msgr2=0x7f128c1e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c1e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 -- 172.21.15.32:0/1509569915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c009420 con 0x7f128c0c26b0 2023-10-16T08:17:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291301640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f128c069e90 0x7f128c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1291b02640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 0x7f128c1e3d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f128400caf0 tx=0x7f128400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128400aca0 con 0x7f128c0c26b0 2023-10-16T08:17:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128400ae00 con 0x7f128c0c26b0 2023-10-16T08:17:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284012620 con 0x7f128c0c26b0 2023-10-16T08:17:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c0c6b60 con 0x7f128c0c26b0 2023-10-16T08:17:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c156b30 con 0x7f128c0c26b0 2023-10-16T08:17:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c0c66e0 con 0x7f128c0c26b0 2023-10-16T08:17:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1284007070 con 0x7f128c0c26b0 2023-10-16T08:17:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.254+0000 7f1281ffb640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12700664a0 0x7f1270068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.258+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f12840841a0 con 0x7f128c0c26b0 2023-10-16T08:17:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.258+0000 7f1291301640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12700664a0 0x7f1270068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.258+0000 7f1291301640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12700664a0 0x7f1270068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12880011f0 tx=0x7f1288009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.258+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f128c0c66e0 con 0x7f128c0c26b0 2023-10-16T08:17:15.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f128c0c66e0 con 0x7f128c0c26b0 2023-10-16T08:17:15.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1281ffb640 1 -- 172.21.15.32:0/1509569915 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f128c0c66e0 con 0x7f128c0c26b0 2023-10-16T08:17:15.424 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12700664a0 msgr2=0x7f1270068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1293565640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12700664a0 0x7f1270068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12880011f0 tx=0x7f1288009040 comp rx=0 tx=0).stop 2023-10-16T08:17:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 msgr2=0x7f128c1e3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1293565640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 0x7f128c1e3d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f128400caf0 tx=0x7f128400a060 comp rx=0 tx=0).stop 2023-10-16T08:17:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.418+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 shutdown_connections 2023-10-16T08:17:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f128c0c26b0 0x7f128c1e3d00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f128c06d010 0x7f128c1e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12700664a0 0x7f1270068960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 --2- 172.21.15.32:0/1509569915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f128c069e90 0x7f128c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 >> 172.21.15.32:0/1509569915 conn(0x7f128c0b0da0 msgr2=0x7f128c0c99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 shutdown_connections 2023-10-16T08:17:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.422+0000 7f1293565640 1 -- 172.21.15.32:0/1509569915 wait complete. 2023-10-16T08:17:15.460 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2023-10-16T08:17:15.461 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T08:17:15.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:15.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cec17640 1 Processor -- start 2023-10-16T08:17:15.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cec17640 1 -- start start 2023-10-16T08:17:15.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cec17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d3010 0x7fd0c80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cec17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c80ce770 con 0x7fd0c80d15d0 2023-10-16T08:17:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cec17640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c80d0600 con 0x7fd0c815cb90 2023-10-16T08:17:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cec17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80c6450 con 0x7fd0c80d3010 2023-10-16T08:17:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0c7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0c80d15d0 0x7fd0c80c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45410/0 (socket says 172.21.15.32:45410) 2023-10-16T08:17:15.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0c7fff640 1 -- 172.21.15.32:0/2460175466 learned_addr learned my addr 172.21.15.32:0/2460175466 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:15.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cd1b4640 1 --2- 172.21.15.32:0/2460175466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d3010 0x7fd0c80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502102061 0 0) 0x7fd0c80ce770 con 0x7fd0c80d15d0 2023-10-16T08:17:15.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b8003530 con 0x7fd0c80d15d0 2023-10-16T08:17:15.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772557418 0 0) 0x7fd0c80d0600 con 0x7fd0c815cb90 2023-10-16T08:17:15.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c80ce770 con 0x7fd0c815cb90 2023-10-16T08:17:15.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/2460175466 >> v1:172.21.15.186:6789/0 conn(0x7fd0c815cb90 legacy=0x7fd0c80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/2460175466 >> v1:172.21.15.32:6789/0 conn(0x7fd0c80d15d0 legacy=0x7fd0c80c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.602+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/2460175466 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80d08a0 con 0x7fd0c80d3010 2023-10-16T08:17:15.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cd1b4640 1 --2- 172.21.15.32:0/2460175466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d3010 0x7fd0c80cfb10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009dc0 tx=0x7fd0c001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb7d02072a41f29 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00a5970 con 0x7fd0c80d3010 2023-10-16T08:17:15.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c00a5ad0 con 0x7fd0c80d3010 2023-10-16T08:17:15.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0c77fe640 1 -- 172.21.15.32:0/2460175466 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00a5e60 con 0x7fd0c80d3010 2023-10-16T08:17:15.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 -- 172.21.15.32:0/2460175466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d3010 msgr2=0x7fd0c80cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 --2- 172.21.15.32:0/2460175466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d3010 0x7fd0c80cfb10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009dc0 tx=0x7fd0c001c8b0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 -- 172.21.15.32:0/2460175466 shutdown_connections 2023-10-16T08:17:15.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 --2- 172.21.15.32:0/2460175466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d3010 0x7fd0c80cfb10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 -- 172.21.15.32:0/2460175466 >> 172.21.15.32:0/2460175466 conn(0x7fd0c809bc50 msgr2=0x7fd0c80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:15.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 -- 172.21.15.32:0/2460175466 shutdown_connections 2023-10-16T08:17:15.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.606+0000 7fd0cec17640 1 -- 172.21.15.32:0/2460175466 wait complete. 2023-10-16T08:17:15.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 Processor -- start 2023-10-16T08:17:15.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 -- start start 2023-10-16T08:17:15.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d15d0 0x7fd0c80caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c80d3010 0x7fd0c8153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 0x7fd0c806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80ca6b0 con 0x7fd0c815cb90 2023-10-16T08:17:15.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80c9a70 con 0x7fd0c80d3010 2023-10-16T08:17:15.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80cb010 con 0x7fd0c80d15d0 2023-10-16T08:17:15.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 0x7fd0c806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 0x7fd0c806b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43042/0 (socket says 172.21.15.32:43042) 2023-10-16T08:17:15.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/283543808 learned_addr learned my addr 172.21.15.32:0/283543808 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:15.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0c7fff640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c80d3010 0x7fd0c8153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cc9b3640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d15d0 0x7fd0c80caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d15d0 msgr2=0x7fd0c80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d15d0 0x7fd0c80caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c80d3010 msgr2=0x7fd0c8153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c80d3010 0x7fd0c8153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 -- 172.21.15.32:0/283543808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0009a70 con 0x7fd0c815cb90 2023-10-16T08:17:15.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cc9b3640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d15d0 0x7fd0c80caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:15.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cd1b4640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 0x7fd0c806b760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00194c0 tx=0x7fd0c0019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00adcc0 con 0x7fd0c815cb90 2023-10-16T08:17:15.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c80c9d10 con 0x7fd0c815cb90 2023-10-16T08:17:15.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c00ade40 con 0x7fd0c815cb90 2023-10-16T08:17:15.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b9690 con 0x7fd0c815cb90 2023-10-16T08:17:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.610+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c80ca2d0 con 0x7fd0c815cb90 2023-10-16T08:17:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.614+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c80ca7f0 con 0x7fd0c815cb90 2023-10-16T08:17:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.614+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0c00b97f0 con 0x7fd0c815cb90 2023-10-16T08:17:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.614+0000 7fd0c57fa640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0ac0664a0 0x7fd0ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.614+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd0c011f770 con 0x7fd0c815cb90 2023-10-16T08:17:15.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.614+0000 7fd0cc9b3640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0ac0664a0 0x7fd0ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.618+0000 7fd0cc9b3640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0ac0664a0 0x7fd0ac068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc001240 tx=0x7fd0bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.618+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0c80ca7f0 con 0x7fd0c815cb90 2023-10-16T08:17:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.782+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0c80ca7f0 con 0x7fd0c815cb90 2023-10-16T08:17:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.782+0000 7fd0c57fa640 1 -- 172.21.15.32:0/283543808 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fd0c80ca7f0 con 0x7fd0c815cb90 2023-10-16T08:17:15.788 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:15.788 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.782+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0ac0664a0 msgr2=0x7fd0ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.782+0000 7fd0cec17640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0ac0664a0 0x7fd0ac068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc001240 tx=0x7fd0bc009040 comp rx=0 tx=0).stop 2023-10-16T08:17:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.782+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 msgr2=0x7fd0c806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.782+0000 7fd0cec17640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 0x7fd0c806b760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00194c0 tx=0x7fd0c0019650 comp rx=0 tx=0).stop 2023-10-16T08:17:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 shutdown_connections 2023-10-16T08:17:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c815cb90 0x7fd0c806b760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c80d3010 0x7fd0c8153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0ac0664a0 0x7fd0ac068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 --2- 172.21.15.32:0/283543808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0c80d15d0 0x7fd0c80caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 >> 172.21.15.32:0/283543808 conn(0x7fd0c809bc50 msgr2=0x7fd0c815bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 shutdown_connections 2023-10-16T08:17:15.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.786+0000 7fd0cec17640 1 -- 172.21.15.32:0/283543808 wait complete. 2023-10-16T08:17:15.809 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:15.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 Processor -- start 2023-10-16T08:17:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 -- start start 2023-10-16T08:17:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5ec0d0210 0x7fa5ec0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec0b84e0 0x7fa5ec0b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec0d1670 con 0x7fa5ec15cba0 2023-10-16T08:17:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0d1460 con 0x7fa5ec0d0210 2023-10-16T08:17:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5f0c9a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0d1950 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5ec0d0210 0x7fa5ec0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5ec0d0210 0x7fa5ec0d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43058/0 (socket says 172.21.15.32:43058) 2023-10-16T08:17:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 -- 172.21.15.32:0/2230630989 learned_addr learned my addr 172.21.15.32:0/2230630989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5ead9d640 1 --2- 172.21.15.32:0/2230630989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec0b84e0 0x7fa5ec0b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 -- 172.21.15.32:0/2230630989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec0b84e0 msgr2=0x7fa5ec0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 --2- 172.21.15.32:0/2230630989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec0b84e0 0x7fa5ec0b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 -- 172.21.15.32:0/2230630989 >> v1:172.21.15.32:6789/0 conn(0x7fa5ec15cba0 legacy=0x7fa5ec0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.954+0000 7fa5e3fff640 1 -- 172.21.15.32:0/2230630989 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0d1c30 con 0x7fa5ec0d0210 2023-10-16T08:17:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5e3fff640 1 --2- 172.21.15.32:0/2230630989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0d0f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009870 tx=0x7fa5d401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562b9574b61571cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5e9d9b640 1 -- 172.21.15.32:0/2230630989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40a9400 con 0x7fa5ec0d0210 2023-10-16T08:17:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5e9d9b640 1 -- 172.21.15.32:0/2230630989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d40a9560 con 0x7fa5ec0d0210 2023-10-16T08:17:15.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5e9d9b640 1 -- 172.21.15.32:0/2230630989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40a98f0 con 0x7fa5ec0d0210 2023-10-16T08:17:15.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/2230630989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0d0210 msgr2=0x7fa5ec0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/2230630989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0d0f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009870 tx=0x7fa5d401c540 comp rx=0 tx=0).stop 2023-10-16T08:17:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/2230630989 shutdown_connections 2023-10-16T08:17:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/2230630989 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec0b84e0 0x7fa5ec0b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/2230630989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0d0f20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/2230630989 >> 172.21.15.32:0/2230630989 conn(0x7fa5ec0719c0 msgr2=0x7fa5ec071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/2230630989 shutdown_connections 2023-10-16T08:17:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/2230630989 wait complete. 2023-10-16T08:17:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 Processor -- start 2023-10-16T08:17:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.958+0000 7fa5f0c9a640 1 -- start start 2023-10-16T08:17:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 0x7fa5ec1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec15cba0 0x7fa5ec158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c78a0 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c8200 con 0x7fa5ec0d0210 2023-10-16T08:17:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c6c20 con 0x7fa5ec15cba0 2023-10-16T08:17:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 0x7fa5ec1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 0x7fa5ec1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43064/0 (socket says 172.21.15.32:43064) 2023-10-16T08:17:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 -- 172.21.15.32:0/1305538386 learned_addr learned my addr 172.21.15.32:0/1305538386 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5e3fff640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ead9d640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec15cba0 0x7fa5ec158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 -- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec15cba0 msgr2=0x7fa5ec158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec15cba0 0x7fa5ec158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 -- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5ec0d0210 msgr2=0x7fa5ec0c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 -- 172.21.15.32:0/1305538386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40094d0 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ead9d640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec15cba0 0x7fa5ec158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5ea59c640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 0x7fa5ec1503d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00c3f0 tx=0x7fa5dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0052d0 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc005430 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc003840 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec0c6f00 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec0c74c0 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.962+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec04e170 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.966+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa5dc0039a0 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.966+0000 7fa5e2ffd640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa5cc0664a0 0x7fa5cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.966+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa5dc0837a0 con 0x7fa5ec0b84e0 2023-10-16T08:17:15.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.966+0000 7fa5e3fff640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa5cc0664a0 0x7fa5cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.966+0000 7fa5e3fff640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa5cc0664a0 0x7fa5cc068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40ae000 tx=0x7fa5d40b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:15.970+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa5dc050870 con 0x7fa5ec0b84e0 2023-10-16T08:17:16.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.126+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5ec156360 con 0x7fa5ec0b84e0 2023-10-16T08:17:16.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.126+0000 7fa5e2ffd640 1 -- 172.21.15.32:0/1305538386 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5dc053e90 con 0x7fa5ec0b84e0 2023-10-16T08:17:16.134 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:16.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa5cc0664a0 msgr2=0x7fa5cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa5cc0664a0 0x7fa5cc068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40ae000 tx=0x7fa5d40b3040 comp rx=0 tx=0).stop 2023-10-16T08:17:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 msgr2=0x7fa5ec1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 0x7fa5ec1503d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00c3f0 tx=0x7fa5dc00c8b0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 shutdown_connections 2023-10-16T08:17:16.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa5cc0664a0 0x7fa5cc068960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa5ec15cba0 0x7fa5ec158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5ec0d0210 0x7fa5ec0c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 --2- 172.21.15.32:0/1305538386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5ec0b84e0 0x7fa5ec1503d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 >> 172.21.15.32:0/1305538386 conn(0x7fa5ec0719c0 msgr2=0x7fa5ec154550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:16.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 shutdown_connections 2023-10-16T08:17:16.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.130+0000 7fa5f0c9a640 1 -- 172.21.15.32:0/1305538386 wait complete. 2023-10-16T08:17:16.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.294+0000 7fb86f1f8640 1 Processor -- start 2023-10-16T08:17:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.294+0000 7fb86f1f8640 1 -- start start 2023-10-16T08:17:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86f1f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb86815cba0 0x7fb8680d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86f1f8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680d1670 con 0x7fb8680d0210 2023-10-16T08:17:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86f1f8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680d1950 con 0x7fb8680b84e0 2023-10-16T08:17:16.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86f1f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680d1460 con 0x7fb86815cba0 2023-10-16T08:17:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86d795640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8680d0210 0x7fb8680b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45426/0 (socket says 172.21.15.32:45426) 2023-10-16T08:17:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86d795640 1 -- 172.21.15.32:0/2619735806 learned_addr learned my addr 172.21.15.32:0/2619735806 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86cf94640 1 --2- 172.21.15.32:0/2619735806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb86815cba0 0x7fb8680d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068314779 0 0) 0x7fb8680d1670 con 0x7fb8680d0210 2023-10-16T08:17:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb858003530 con 0x7fb8680d0210 2023-10-16T08:17:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504863852 0 0) 0x7fb8680d1950 con 0x7fb8680b84e0 2023-10-16T08:17:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8680d1670 con 0x7fb8680b84e0 2023-10-16T08:17:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86cf94640 1 -- 172.21.15.32:0/2619735806 >> v1:172.21.15.186:6789/0 conn(0x7fb8680b84e0 legacy=0x7fb8680cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86cf94640 1 -- 172.21.15.32:0/2619735806 >> v1:172.21.15.32:6789/0 conn(0x7fb8680d0210 legacy=0x7fb8680b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.298+0000 7fb86cf94640 1 -- 172.21.15.32:0/2619735806 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680d1c30 con 0x7fb86815cba0 2023-10-16T08:17:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86cf94640 1 --2- 172.21.15.32:0/2619735806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb86815cba0 0x7fb8680d0f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0097f0 tx=0x7fb85c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faae7ca5c03ccd42 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0aa620 con 0x7fb86815cba0 2023-10-16T08:17:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c0aa780 con 0x7fb86815cba0 2023-10-16T08:17:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb8677fe640 1 -- 172.21.15.32:0/2619735806 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0aab10 con 0x7fb86815cba0 2023-10-16T08:17:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 -- 172.21.15.32:0/2619735806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb86815cba0 msgr2=0x7fb8680d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/2619735806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb86815cba0 0x7fb8680d0f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0097f0 tx=0x7fb85c01c4c0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 -- 172.21.15.32:0/2619735806 shutdown_connections 2023-10-16T08:17:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/2619735806 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb86815cba0 0x7fb8680d0f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 -- 172.21.15.32:0/2619735806 >> 172.21.15.32:0/2619735806 conn(0x7fb8680719c0 msgr2=0x7fb868071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 -- 172.21.15.32:0/2619735806 shutdown_connections 2023-10-16T08:17:16.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 -- 172.21.15.32:0/2619735806 wait complete. 2023-10-16T08:17:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 Processor -- start 2023-10-16T08:17:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.302+0000 7fb86f1f8640 1 -- start start 2023-10-16T08:17:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8680b84e0 0x7fb86814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8680d0210 0x7fb8680c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 0x7fb8680c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8681515b0 con 0x7fb86815cba0 2023-10-16T08:17:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86814f970 con 0x7fb8680b84e0 2023-10-16T08:17:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb868150930 con 0x7fb8680d0210 2023-10-16T08:17:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 0x7fb8680c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 0x7fb8680c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43076/0 (socket says 172.21.15.32:43076) 2023-10-16T08:17:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 -- 172.21.15.32:0/482889894 learned_addr learned my addr 172.21.15.32:0/482889894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86cf94640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8680b84e0 0x7fb86814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb867fff640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8680d0210 0x7fb8680c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 -- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8680d0210 msgr2=0x7fb8680c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8680d0210 0x7fb8680c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 -- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8680b84e0 msgr2=0x7fb86814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8680b84e0 0x7fb86814f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 -- 172.21.15.32:0/482889894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c009420 con 0x7fb86815cba0 2023-10-16T08:17:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86d795640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 0x7fb8680c7190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb86000cba0 tx=0x7fb86000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000ace0 con 0x7fb86815cba0 2023-10-16T08:17:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86000ae40 con 0x7fb86815cba0 2023-10-16T08:17:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860012600 con 0x7fb86815cba0 2023-10-16T08:17:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb868150c10 con 0x7fb86815cba0 2023-10-16T08:17:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8681511d0 con 0x7fb86815cba0 2023-10-16T08:17:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.306+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86804e170 con 0x7fb86815cba0 2023-10-16T08:17:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.310+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb860007070 con 0x7fb86815cba0 2023-10-16T08:17:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.310+0000 7fb8657fa640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb84c0664a0 0x7fb84c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.310+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb860084190 con 0x7fb86815cba0 2023-10-16T08:17:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.310+0000 7fb86cf94640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb84c0664a0 0x7fb84c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.310+0000 7fb86cf94640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb84c0664a0 0x7fb84c068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0ae000 tx=0x7fb85c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.314+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb860051260 con 0x7fb86815cba0 2023-10-16T08:17:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.470+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8680d38c0 con 0x7fb86815cba0 2023-10-16T08:17:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.470+0000 7fb8657fa640 1 -- 172.21.15.32:0/482889894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb860054880 con 0x7fb86815cba0 2023-10-16T08:17:16.476 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb84c0664a0 msgr2=0x7fb84c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb84c0664a0 0x7fb84c068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0ae000 tx=0x7fb85c0a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 msgr2=0x7fb8680c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 0x7fb8680c7190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb86000cba0 tx=0x7fb86000a430 comp rx=0 tx=0).stop 2023-10-16T08:17:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 shutdown_connections 2023-10-16T08:17:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb86815cba0 0x7fb8680c7190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb84c0664a0 0x7fb84c068960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb8680d0210 0x7fb8680c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 --2- 172.21.15.32:0/482889894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8680b84e0 0x7fb86814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 >> 172.21.15.32:0/482889894 conn(0x7fb8680719c0 msgr2=0x7fb868071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 shutdown_connections 2023-10-16T08:17:16.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.474+0000 7fb86f1f8640 1 -- 172.21.15.32:0/482889894 wait complete. 2023-10-16T08:17:16.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff0879d9640 1 Processor -- start 2023-10-16T08:17:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff0879d9640 1 -- start start 2023-10-16T08:17:16.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff0879d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 0x7ff0800d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff0879d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0800d1670 con 0x7ff08015cba0 2023-10-16T08:17:16.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff0879d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0800d1950 con 0x7ff0800d0210 2023-10-16T08:17:16.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff0879d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800d1460 con 0x7ff0800b84e0 2023-10-16T08:17:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff085775640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff08015cba0 0x7ff0800cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45438/0 (socket says 172.21.15.32:45438) 2023-10-16T08:17:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff085775640 1 -- 172.21.15.32:0/1419064136 learned_addr learned my addr 172.21.15.32:0/1419064136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff085f76640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 0x7ff0800d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325592115 0 0) 0x7ff0800d1670 con 0x7ff08015cba0 2023-10-16T08:17:16.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff06c003530 con 0x7ff08015cba0 2023-10-16T08:17:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109838704 0 0) 0x7ff0800d1950 con 0x7ff0800d0210 2023-10-16T08:17:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0800d1670 con 0x7ff0800d0210 2023-10-16T08:17:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971612063 0 0) 0x7ff06c003530 con 0x7ff08015cba0 2023-10-16T08:17:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 msgr2=0x7ff0800d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 0x7ff0800d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 >> v1:172.21.15.186:6789/0 conn(0x7ff0800d0210 legacy=0x7ff0800b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800d1c30 con 0x7ff08015cba0 2023-10-16T08:17:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7ff07c001600 con 0x7ff08015cba0 2023-10-16T08:17:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 >> v1:172.21.15.32:6789/0 conn(0x7ff08015cba0 legacy=0x7ff0800cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.638+0000 7ff077fff640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 0x7ff06c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff06c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff06c004d10 0x7ff06c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800d1460 con 0x7ff06c0040f0 2023-10-16T08:17:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c00bad0 con 0x7ff06c004d10 2023-10-16T08:17:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c00bc90 con 0x7ff0800b84e0 2023-10-16T08:17:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff06c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff085f76640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff06c004d10 0x7ff06c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff085775640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 0x7ff06c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 -- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 msgr2=0x7ff06c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff0800b84e0 0x7ff06c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 -- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff06c004d10 msgr2=0x7ff06c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff06c004d10 0x7ff06c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 -- 172.21.15.32:0/1419064136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c003530 con 0x7ff06c0040f0 2023-10-16T08:17:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff084f74640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff06c0047d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0800b5aa0 tx=0x7ff07001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f908a8fd6caebf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700052d0 con 0x7ff06c0040f0 2023-10-16T08:17:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070005450 con 0x7ff06c0040f0 2023-10-16T08:17:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.642+0000 7ff077fff640 1 -- 172.21.15.32:0/1419064136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700a8850 con 0x7ff06c0040f0 2023-10-16T08:17:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- 172.21.15.32:0/1419064136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 msgr2=0x7ff06c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1419064136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff06c0047d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0800b5aa0 tx=0x7ff07001ebf0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085f76640 1 -- 172.21.15.32:0/1419064136 reap_dead start 2023-10-16T08:17:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- 172.21.15.32:0/1419064136 shutdown_connections 2023-10-16T08:17:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- 172.21.15.32:0/1419064136 >> 172.21.15.32:0/1419064136 conn(0x7ff0800719c0 msgr2=0x7ff080071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- 172.21.15.32:0/1419064136 shutdown_connections 2023-10-16T08:17:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- 172.21.15.32:0/1419064136 wait complete. 2023-10-16T08:17:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 Processor -- start 2023-10-16T08:17:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- start start 2023-10-16T08:17:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff080152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080153650 0x7ff080153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0801f2230 0x7ff0801f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0801544d0 con 0x7ff06c0040f0 2023-10-16T08:17:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0801547b0 con 0x7ff0801f2230 2023-10-16T08:17:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff0879d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff080154a90 con 0x7ff080153650 2023-10-16T08:17:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff080152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff080152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43096/0 (socket says 172.21.15.32:43096) 2023-10-16T08:17:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 -- 172.21.15.32:0/1878677249 learned_addr learned my addr 172.21.15.32:0/1878677249 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085f76640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0801f2230 0x7ff0801f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff084f74640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080153650 0x7ff080153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 -- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080153650 msgr2=0x7ff080153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080153650 0x7ff080153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 -- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0801f2230 msgr2=0x7ff0801f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0801f2230 0x7ff0801f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085775640 1 -- 172.21.15.32:0/1878677249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080154d70 con 0x7ff06c0040f0 2023-10-16T08:17:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.646+0000 7ff085f76640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0801f2230 0x7ff0801f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff085775640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff080152f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0072c0 tx=0x7ff07c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00f7b0 con 0x7ff06c0040f0 2023-10-16T08:17:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c00f930 con 0x7ff06c0040f0 2023-10-16T08:17:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080155050 con 0x7ff06c0040f0 2023-10-16T08:17:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00fce0 con 0x7ff06c0040f0 2023-10-16T08:17:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080155570 con 0x7ff06c0040f0 2023-10-16T08:17:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08004e170 con 0x7ff06c0040f0 2023-10-16T08:17:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff07c01d460 con 0x7ff06c0040f0 2023-10-16T08:17:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff075ffb640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0640664d0 0x7ff064068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff07c084340 con 0x7ff06c0040f0 2023-10-16T08:17:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.650+0000 7ff084f74640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0640664d0 0x7ff064068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.654+0000 7ff084f74640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0640664d0 0x7ff064068990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0700b0000 tx=0x7ff0700aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.654+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff07c054ea0 con 0x7ff06c0040f0 2023-10-16T08:17:16.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0800ce3e0 con 0x7ff06c0040f0 2023-10-16T08:17:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff075ffb640 1 -- 172.21.15.32:0/1878677249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff07c054a30 con 0x7ff06c0040f0 2023-10-16T08:17:16.819 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0640664d0 msgr2=0x7ff064068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0640664d0 0x7ff064068990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0700b0000 tx=0x7ff0700aa070 comp rx=0 tx=0).stop 2023-10-16T08:17:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 msgr2=0x7ff080152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff080152f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0072c0 tx=0x7ff07c007780 comp rx=0 tx=0).stop 2023-10-16T08:17:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 shutdown_connections 2023-10-16T08:17:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0801f2230 0x7ff0801f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff0640664d0 0x7ff064068990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff080153650 0x7ff080153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 --2- 172.21.15.32:0/1878677249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff06c0040f0 0x7ff080152f70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 >> 172.21.15.32:0/1878677249 conn(0x7ff0800719c0 msgr2=0x7ff0800d2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 shutdown_connections 2023-10-16T08:17:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.814+0000 7ff0879d9640 1 -- 172.21.15.32:0/1878677249 wait complete. 2023-10-16T08:17:16.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 Processor -- start 2023-10-16T08:17:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 -- start start 2023-10-16T08:17:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe57806a0f0 0x7fe5780c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5780ade00 0x7fe5780ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57806c6d0 con 0x7fe57806a4d0 2023-10-16T08:17:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57806c950 con 0x7fe57806a0f0 2023-10-16T08:17:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57f8ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781654f0 con 0x7fe5780ade00 2023-10-16T08:17:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57de87640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5780ade00 0x7fe5780ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57d686640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe57806a0f0 0x7fe5780c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57ce85640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fe57806a4d0 0x7fe5780ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:57096/0 (socket says 172.21.15.32:57096) 2023-10-16T08:17:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57ce85640 1 -- 172.21.15.32:0/2244505666 learned_addr learned my addr 172.21.15.32:0/2244505666 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe56ffff640 1 -- 172.21.15.32:0/2244505666 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222753503 0 0) 0x7fe57806c6d0 con 0x7fe57806a4d0 2023-10-16T08:17:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe56ffff640 1 -- 172.21.15.32:0/2244505666 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe564003530 con 0x7fe57806a4d0 2023-10-16T08:17:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57de87640 1 -- 172.21.15.32:0/2244505666 >> v2:172.21.15.186:3300/0 conn(0x7fe57806a0f0 msgr2=0x7fe5780c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57de87640 1 --2- 172.21.15.32:0/2244505666 >> v2:172.21.15.186:3300/0 conn(0x7fe57806a0f0 0x7fe5780c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57de87640 1 -- 172.21.15.32:0/2244505666 >> v1:172.21.15.186:6789/0 conn(0x7fe57806a4d0 legacy=0x7fe5780ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57de87640 1 -- 172.21.15.32:0/2244505666 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5781657d0 con 0x7fe5780ade00 2023-10-16T08:17:16.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.982+0000 7fe57d686640 1 --2- 172.21.15.32:0/2244505666 >> v2:172.21.15.186:3300/0 conn(0x7fe57806a0f0 0x7fe5780c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:16.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57de87640 1 --2- 172.21.15.32:0/2244505666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5780ade00 0x7fe5780ccd00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe570009d90 tx=0x7fe57001d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c420d3cefde38081 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:16.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe56ffff640 1 -- 172.21.15.32:0/2244505666 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700a5a30 con 0x7fe5780ade00 2023-10-16T08:17:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe56ffff640 1 -- 172.21.15.32:0/2244505666 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5700a5b90 con 0x7fe5780ade00 2023-10-16T08:17:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe56ffff640 1 -- 172.21.15.32:0/2244505666 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700ad440 con 0x7fe5780ade00 2023-10-16T08:17:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 -- 172.21.15.32:0/2244505666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5780ade00 msgr2=0x7fe5780ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/2244505666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5780ade00 0x7fe5780ccd00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe570009d90 tx=0x7fe57001d3b0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 -- 172.21.15.32:0/2244505666 shutdown_connections 2023-10-16T08:17:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/2244505666 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe5780ade00 0x7fe5780ccd00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/2244505666 >> v2:172.21.15.186:3300/0 conn(0x7fe57806a0f0 0x7fe5780c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 -- 172.21.15.32:0/2244505666 >> 172.21.15.32:0/2244505666 conn(0x7fe5780b0da0 msgr2=0x7fe5780ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 -- 172.21.15.32:0/2244505666 shutdown_connections 2023-10-16T08:17:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.986+0000 7fe57f8ea640 1 -- 172.21.15.32:0/2244505666 wait complete. 2023-10-16T08:17:16.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 Processor -- start 2023-10-16T08:17:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- start start 2023-10-16T08:17:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe57806a0f0 0x7fe5780b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 0x7fe5780c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5780ade00 0x7fe5780c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780b2680 con 0x7fe57806a4d0 2023-10-16T08:17:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780c8750 con 0x7fe5780ade00 2023-10-16T08:17:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780b1a00 con 0x7fe57806a0f0 2023-10-16T08:17:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 0x7fe5780c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 0x7fe5780c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43102/0 (socket says 172.21.15.32:43102) 2023-10-16T08:17:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 -- 172.21.15.32:0/1320378384 learned_addr learned my addr 172.21.15.32:0/1320378384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57d686640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe57806a0f0 0x7fe5780b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57de87640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5780ade00 0x7fe5780c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 -- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe57806a0f0 msgr2=0x7fe5780b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe57806a0f0 0x7fe5780b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 -- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5780ade00 msgr2=0x7fe5780c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5780ade00 0x7fe5780c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 -- 172.21.15.32:0/1320378384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574009420 con 0x7fe57806a4d0 2023-10-16T08:17:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57ce85640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 0x7fe5780c8210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c450 tx=0x7fe56800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568005cf0 con 0x7fe57806a4d0 2023-10-16T08:17:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568005e50 con 0x7fe57806a4d0 2023-10-16T08:17:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe570009a70 con 0x7fe57806a4d0 2023-10-16T08:17:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568007270 con 0x7fe57806a4d0 2023-10-16T08:17:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5780b1d90 con 0x7fe57806a4d0 2023-10-16T08:17:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.990+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5780b27c0 con 0x7fe57806a4d0 2023-10-16T08:17:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.994+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe5680073d0 con 0x7fe57806a4d0 2023-10-16T08:17:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.994+0000 7fe56dffb640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe55c0664a0 0x7fe55c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.994+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe568083700 con 0x7fe57806a4d0 2023-10-16T08:17:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.994+0000 7fe57d686640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe55c0664a0 0x7fe55c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.994+0000 7fe57d686640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe55c0664a0 0x7fe55c068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe574009c50 tx=0x7fe574009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:16.998+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe5780b27c0 con 0x7fe57806a4d0 2023-10-16T08:17:17.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:17:17.142 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:17:17.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.154+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe5780b27c0 con 0x7fe57806a4d0 2023-10-16T08:17:17.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.154+0000 7fe56dffb640 1 -- 172.21.15.32:0/1320378384 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe5780b27c0 con 0x7fe57806a4d0 2023-10-16T08:17:17.161 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:17.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe55c0664a0 msgr2=0x7fe55c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe55c0664a0 0x7fe55c068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe574009c50 tx=0x7fe574009190 comp rx=0 tx=0).stop 2023-10-16T08:17:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 msgr2=0x7fe5780c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 0x7fe5780c8210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c450 tx=0x7fe56800c910 comp rx=0 tx=0).stop 2023-10-16T08:17:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 shutdown_connections 2023-10-16T08:17:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5780ade00 0x7fe5780c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe57806a4d0 0x7fe5780c8210 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe55c0664a0 0x7fe55c068960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 --2- 172.21.15.32:0/1320378384 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe57806a0f0 0x7fe5780b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 >> 172.21.15.32:0/1320378384 conn(0x7fe5780b0da0 msgr2=0x7fe578160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 shutdown_connections 2023-10-16T08:17:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.158+0000 7fe57f8ea640 1 -- 172.21.15.32:0/1320378384 wait complete. 2023-10-16T08:17:17.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-10-16T08:17:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f46816d6640 1 Processor -- start 2023-10-16T08:17:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f46816d6640 1 -- start start 2023-10-16T08:17:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f46816d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f467c0cf020 0x7f467c0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f46816d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0ce730 con 0x7f467c0cdd90 2023-10-16T08:17:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f46816d6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0d0ba0 con 0x7f467c15cae0 2023-10-16T08:17:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f46816d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c0cf9f0 con 0x7f467c0cf020 2023-10-16T08:17:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f467b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f467c0cf020 0x7f467c0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f4673fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f467c0cdd90 0x7f467c0d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45454/0 (socket says 172.21.15.32:45454) 2023-10-16T08:17:17.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f4673fff640 1 -- 172.21.15.32:0/284625991 learned_addr learned my addr 172.21.15.32:0/284625991 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:17.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f467b7fe640 1 -- 172.21.15.32:0/284625991 >> v1:172.21.15.186:6789/0 conn(0x7f467c15cae0 legacy=0x7f467c0d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f467b7fe640 1 -- 172.21.15.32:0/284625991 >> v1:172.21.15.32:6789/0 conn(0x7f467c0cdd90 legacy=0x7f467c0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.374+0000 7f467b7fe640 1 -- 172.21.15.32:0/284625991 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c0ce200 con 0x7f467c0cf020 2023-10-16T08:17:17.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f467b7fe640 1 --2- 172.21.15.32:0/284625991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cf020 0x7f467c0cf4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4674009d90 tx=0x7f467401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cdb7a460730599 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:17.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f467a7fc640 1 -- 172.21.15.32:0/284625991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740a59e0 con 0x7f467c0cf020 2023-10-16T08:17:17.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f467a7fc640 1 -- 172.21.15.32:0/284625991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46740a5b40 con 0x7f467c0cf020 2023-10-16T08:17:17.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f467a7fc640 1 -- 172.21.15.32:0/284625991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740ad440 con 0x7f467c0cf020 2023-10-16T08:17:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 -- 172.21.15.32:0/284625991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cf020 msgr2=0x7f467c0cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 --2- 172.21.15.32:0/284625991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cf020 0x7f467c0cf4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4674009d90 tx=0x7f467401c920 comp rx=0 tx=0).stop 2023-10-16T08:17:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 -- 172.21.15.32:0/284625991 shutdown_connections 2023-10-16T08:17:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 --2- 172.21.15.32:0/284625991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cf020 0x7f467c0cf4b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 -- 172.21.15.32:0/284625991 >> 172.21.15.32:0/284625991 conn(0x7f467c09bb50 msgr2=0x7f467c0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 -- 172.21.15.32:0/284625991 shutdown_connections 2023-10-16T08:17:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.378+0000 7f46816d6640 1 -- 172.21.15.32:0/284625991 wait complete. 2023-10-16T08:17:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 Processor -- start 2023-10-16T08:17:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- start start 2023-10-16T08:17:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 0x7f467c1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f467c0cf020 0x7f467c15f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f467c15cae0 0x7f467c15f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0b5d30 con 0x7f467c0cdd90 2023-10-16T08:17:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c155440 con 0x7f467c0cf020 2023-10-16T08:17:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c15fb70 con 0x7f467c15cae0 2023-10-16T08:17:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 0x7f467c1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 0x7f467c1523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43120/0 (socket says 172.21.15.32:43120) 2023-10-16T08:17:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 -- 172.21.15.32:0/2336164222 learned_addr learned my addr 172.21.15.32:0/2336164222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f4673fff640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f467c0cf020 0x7f467c15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467b7fe640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f467c15cae0 0x7f467c15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 -- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f467c15cae0 msgr2=0x7f467c15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f467c15cae0 0x7f467c15f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 -- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f467c0cf020 msgr2=0x7f467c15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f467c0cf020 0x7f467c15f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 -- 172.21.15.32:0/2336164222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674009a70 con 0x7f467c0cdd90 2023-10-16T08:17:17.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f4673fff640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f467c0cf020 0x7f467c15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:17.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f467affd640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 0x7f467c1523c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c400 tx=0x7f466c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:17.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c0052d0 con 0x7f467c0cdd90 2023-10-16T08:17:17.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c005430 con 0x7f467c0cdd90 2023-10-16T08:17:17.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c003890 con 0x7f467c0cdd90 2023-10-16T08:17:17.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c0b5400 con 0x7f467c0cdd90 2023-10-16T08:17:17.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c0b59c0 con 0x7f467c0cdd90 2023-10-16T08:17:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.382+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c067230 con 0x7f467c0cdd90 2023-10-16T08:17:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.386+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f466c0039f0 con 0x7f467c0cdd90 2023-10-16T08:17:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.386+0000 7f46737fe640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f465c0664a0 0x7f465c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.386+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f466c083880 con 0x7f467c0cdd90 2023-10-16T08:17:17.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.386+0000 7f4673fff640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f465c0664a0 0x7f465c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:17.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.386+0000 7f4673fff640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f465c0664a0 0x7f465c068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f467c15ef30 tx=0x7f4664009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:17.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.390+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f466c050950 con 0x7f467c0cdd90 2023-10-16T08:17:17.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.602+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f467c0d2cd0 con 0x7f467c0cdd90 2023-10-16T08:17:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46737fe640 1 -- 172.21.15.32:0/2336164222 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v76) v1 ==== 150+0+0 (secure 0 0 0) 0x7f466c053f70 con 0x7f467c0cdd90 2023-10-16T08:17:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f465c0664a0 msgr2=0x7f465c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f465c0664a0 0x7f465c068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f467c15ef30 tx=0x7f4664009040 comp rx=0 tx=0).stop 2023-10-16T08:17:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 msgr2=0x7f467c1523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 0x7f467c1523c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c400 tx=0x7f466c00c8c0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 shutdown_connections 2023-10-16T08:17:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f465c0664a0 0x7f465c068960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f467c15cae0 0x7f467c15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f467c0cf020 0x7f467c15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 --2- 172.21.15.32:0/2336164222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c0cdd90 0x7f467c1523c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 >> 172.21.15.32:0/2336164222 conn(0x7f467c09bb50 msgr2=0x7f467c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.914+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 shutdown_connections 2023-10-16T08:17:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:17.918+0000 7f46816d6640 1 -- 172.21.15.32:0/2336164222 wait complete. 2023-10-16T08:17:17.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T08:17:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 Processor -- start 2023-10-16T08:17:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 -- start start 2023-10-16T08:17:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c069e90 0x7fca6c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca6c06a270 0x7fca6c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6c15ae30 con 0x7fca6c0702f0 2023-10-16T08:17:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c0cb090 con 0x7fca6c06a270 2023-10-16T08:17:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca71565640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0ca510 con 0x7fca6c069e90 2023-10-16T08:17:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.086+0000 7fca63fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca6c06a270 0x7fca6c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca6c06a270 0x7fca6c0c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43132/0 (socket says 172.21.15.32:43132) 2023-10-16T08:17:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 -- 172.21.15.32:0/3779616142 learned_addr learned my addr 172.21.15.32:0/3779616142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca6affd640 1 --2- 172.21.15.32:0/3779616142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c069e90 0x7fca6c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 -- 172.21.15.32:0/3779616142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c069e90 msgr2=0x7fca6c06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 --2- 172.21.15.32:0/3779616142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c069e90 0x7fca6c06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 -- 172.21.15.32:0/3779616142 >> v1:172.21.15.32:6789/0 conn(0x7fca6c0702f0 legacy=0x7fca6c070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 -- 172.21.15.32:0/3779616142 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c0ca7f0 con 0x7fca6c06a270 2023-10-16T08:17:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca63fff640 1 --2- 172.21.15.32:0/3779616142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0c9ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca54009870 tx=0x7fca5401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fac2d2e42e543ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca6a7fc640 1 -- 172.21.15.32:0/3779616142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540aa670 con 0x7fca6c06a270 2023-10-16T08:17:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca6a7fc640 1 -- 172.21.15.32:0/3779616142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca540aa7d0 con 0x7fca6c06a270 2023-10-16T08:17:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.090+0000 7fca6a7fc640 1 -- 172.21.15.32:0/3779616142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540aab60 con 0x7fca6c06a270 2023-10-16T08:17:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- 172.21.15.32:0/3779616142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 msgr2=0x7fca6c0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 --2- 172.21.15.32:0/3779616142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0c9ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca54009870 tx=0x7fca5401c510 comp rx=0 tx=0).stop 2023-10-16T08:17:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- 172.21.15.32:0/3779616142 shutdown_connections 2023-10-16T08:17:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 --2- 172.21.15.32:0/3779616142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0c9ce0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 --2- 172.21.15.32:0/3779616142 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c069e90 0x7fca6c06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- 172.21.15.32:0/3779616142 >> 172.21.15.32:0/3779616142 conn(0x7fca6c0b0da0 msgr2=0x7fca6c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- 172.21.15.32:0/3779616142 shutdown_connections 2023-10-16T08:17:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- 172.21.15.32:0/3779616142 wait complete. 2023-10-16T08:17:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 Processor -- start 2023-10-16T08:17:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- start start 2023-10-16T08:17:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c069e90 0x7fca6c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0702f0 0x7fca6c1f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0cc890 con 0x7fca6c06a270 2023-10-16T08:17:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1f9f00 con 0x7fca6c069e90 2023-10-16T08:17:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca71565640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1f5930 con 0x7fca6c0702f0 2023-10-16T08:17:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43138/0 (socket says 172.21.15.32:43138) 2023-10-16T08:17:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 -- 172.21.15.32:0/1036466863 learned_addr learned my addr 172.21.15.32:0/1036466863 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca6affd640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c069e90 0x7fca6c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca6b7fe640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0702f0 0x7fca6c1f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 -- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0702f0 msgr2=0x7fca6c1f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0702f0 0x7fca6c1f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 -- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c069e90 msgr2=0x7fca6c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c069e90 0x7fca6c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca63fff640 1 -- 172.21.15.32:0/1036466863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540094d0 con 0x7fca6c06a270 2023-10-16T08:17:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.094+0000 7fca6b7fe640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0702f0 0x7fca6c1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca63fff640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0cc350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca540ac040 tx=0x7fca5401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:18.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540b3800 con 0x7fca6c06a270 2023-10-16T08:17:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca540b3980 con 0x7fca6c06a270 2023-10-16T08:17:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540b3d30 con 0x7fca6c06a270 2023-10-16T08:17:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c1f5bd0 con 0x7fca6c06a270 2023-10-16T08:17:18.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c0ccd60 con 0x7fca6c06a270 2023-10-16T08:17:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c1f5d60 con 0x7fca6c06a270 2023-10-16T08:17:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca540b2380 con 0x7fca6c06a270 2023-10-16T08:17:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca637fe640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fca4c0664d0 0x7fca4c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fca5411eb80 con 0x7fca6c06a270 2023-10-16T08:17:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.098+0000 7fca6affd640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fca4c0664d0 0x7fca4c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.102+0000 7fca6affd640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fca4c0664d0 0x7fca4c068990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b9d0 tx=0x7fca5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.102+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fca540ebc50 con 0x7fca6c06a270 2023-10-16T08:17:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.262+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fca6c04e170 con 0x7fca6c06a270 2023-10-16T08:17:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca637fe640 1 -- 172.21.15.32:0/1036466863 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca540c3020 con 0x7fca6c06a270 2023-10-16T08:17:18.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fca4c0664d0 msgr2=0x7fca4c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fca4c0664d0 0x7fca4c068990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b9d0 tx=0x7fca5c008040 comp rx=0 tx=0).stop 2023-10-16T08:17:18.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 msgr2=0x7fca6c0cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0cc350 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca540ac040 tx=0x7fca5401ad50 comp rx=0 tx=0).stop 2023-10-16T08:17:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 shutdown_connections 2023-10-16T08:17:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fca4c0664d0 0x7fca4c068990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fca6c0702f0 0x7fca6c1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca6c06a270 0x7fca6c0cc350 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 --2- 172.21.15.32:0/1036466863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca6c069e90 0x7fca6c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 >> 172.21.15.32:0/1036466863 conn(0x7fca6c0b0da0 msgr2=0x7fca6c159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 shutdown_connections 2023-10-16T08:17:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.266+0000 7fca71565640 1 -- 172.21.15.32:0/1036466863 wait complete. 2023-10-16T08:17:18.292 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T08:17:18.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:17:18.396 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:17:18.501 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T08:17:18.501 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:17:18.502 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:18.512 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:17:18.512 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:18.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 Processor -- start 2023-10-16T08:17:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 -- start start 2023-10-16T08:17:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f82dc0d15d0 0x7f82dc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 0x7f82dc0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc0ce770 con 0x7f82dc15cb90 2023-10-16T08:17:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc0d0600 con 0x7f82dc0d15d0 2023-10-16T08:17:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82e0c17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c6450 con 0x7f82dc0d3010 2023-10-16T08:17:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82da59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f82dc15cb90 0x7f82dc0d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45468/0 (socket says 172.21.15.32:45468) 2023-10-16T08:17:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82da59c640 1 -- 172.21.15.32:0/4030137700 learned_addr learned my addr 172.21.15.32:0/4030137700 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82dad9d640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 0x7f82dc0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d3fff640 1 --2- 172.21.15.32:0/4030137700 >> v2:172.21.15.186:3300/0 conn(0x7f82dc0d15d0 0x7f82dc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468750203 0 0) 0x7f82dc0ce770 con 0x7f82dc15cb90 2023-10-16T08:17:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c8003530 con 0x7f82dc15cb90 2023-10-16T08:17:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782373641 0 0) 0x7f82c8003530 con 0x7f82dc15cb90 2023-10-16T08:17:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 msgr2=0x7f82dc0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 0x7f82dc0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 >> v2:172.21.15.186:3300/0 conn(0x7f82dc0d15d0 msgr2=0x7f82dc0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 --2- 172.21.15.32:0/4030137700 >> v2:172.21.15.186:3300/0 conn(0x7f82dc0d15d0 0x7f82dc0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0d08a0 con 0x7f82dc15cb90 2023-10-16T08:17:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f82cc001800 con 0x7f82dc15cb90 2023-10-16T08:17:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 >> v1:172.21.15.32:6789/0 conn(0x7f82dc15cb90 legacy=0x7f82dc0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82c8003980 0x7f82c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c80047d0 0x7f82c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 0x7f82c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0d0600 con 0x7f82c8003980 2023-10-16T08:17:18.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c6450 con 0x7f82c80047d0 2023-10-16T08:17:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c800b990 con 0x7f82dc0d3010 2023-10-16T08:17:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82da59c640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82c8003980 0x7f82c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82d3fff640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c80047d0 0x7f82c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.750+0000 7f82dad9d640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 0x7f82c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82da59c640 1 -- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 msgr2=0x7f82c8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82da59c640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0d3010 0x7f82c8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82da59c640 1 -- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c80047d0 msgr2=0x7f82c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82da59c640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c80047d0 0x7f82c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82da59c640 1 -- 172.21.15.32:0/4030137700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c8003530 con 0x7f82c8003980 2023-10-16T08:17:18.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82da59c640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82c8003980 0x7f82c8003da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c270 tx=0x7f82cc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346606115a996e9b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:18.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0070d0 con 0x7f82c8003980 2023-10-16T08:17:18.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc007230 con 0x7f82c8003980 2023-10-16T08:17:18.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82d9d9b640 1 -- 172.21.15.32:0/4030137700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0075e0 con 0x7f82c8003980 2023-10-16T08:17:18.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82e0c17640 1 -- 172.21.15.32:0/4030137700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82c8003980 msgr2=0x7f82c8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82e0c17640 1 --2- 172.21.15.32:0/4030137700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82c8003980 0x7f82c8003da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c270 tx=0x7f82cc01cc30 comp rx=0 tx=0).stop 2023-10-16T08:17:18.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82dad9d640 1 -- 172.21.15.32:0/4030137700 reap_dead start 2023-10-16T08:17:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82e0c17640 1 -- 172.21.15.32:0/4030137700 shutdown_connections 2023-10-16T08:17:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82e0c17640 1 -- 172.21.15.32:0/4030137700 >> 172.21.15.32:0/4030137700 conn(0x7f82dc09bc50 msgr2=0x7f82dc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82e0c17640 1 -- 172.21.15.32:0/4030137700 shutdown_connections 2023-10-16T08:17:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.754+0000 7f82e0c17640 1 -- 172.21.15.32:0/4030137700 wait complete. 2023-10-16T08:17:18.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 Processor -- start 2023-10-16T08:17:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 -- start start 2023-10-16T08:17:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c8003980 0x7f82dc150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0b6510 0x7f82dc150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 0x7f82dc0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0cf890 con 0x7f82dc0ceed0 2023-10-16T08:17:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b7c90 con 0x7f82c8003980 2023-10-16T08:17:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82e0c17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b7800 con 0x7f82dc0b6510 2023-10-16T08:17:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 0x7f82dc0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 0x7f82dc0cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43164/0 (socket says 172.21.15.32:43164) 2023-10-16T08:17:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 -- 172.21.15.32:0/3511818598 learned_addr learned my addr 172.21.15.32:0/3511818598 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82da59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c8003980 0x7f82dc150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82d3fff640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0b6510 0x7f82dc150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 -- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0b6510 msgr2=0x7f82dc150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0b6510 0x7f82dc150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 -- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c8003980 msgr2=0x7f82dc150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c8003980 0x7f82dc150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 -- 172.21.15.32:0/3511818598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0b6990 con 0x7f82dc0ceed0 2023-10-16T08:17:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.758+0000 7f82dad9d640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 0x7f82dc0cf350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82d4011c80 tx=0x7f82d400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400e040 con 0x7f82dc0ceed0 2023-10-16T08:17:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d400f040 con 0x7f82dc0ceed0 2023-10-16T08:17:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4015670 con 0x7f82dc0ceed0 2023-10-16T08:17:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc0b6bf0 con 0x7f82dc0ceed0 2023-10-16T08:17:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc0b71b0 con 0x7f82dc0ceed0 2023-10-16T08:17:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc0b7990 con 0x7f82dc0ceed0 2023-10-16T08:17:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82d40157d0 con 0x7f82dc0ceed0 2023-10-16T08:17:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82d2ffd640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f82bc0664d0 0x7f82bc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f82d40848c0 con 0x7f82dc0ceed0 2023-10-16T08:17:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.762+0000 7f82da59c640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f82bc0664d0 0x7f82bc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.766+0000 7f82da59c640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f82bc0664d0 0x7f82bc068990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0aa000 tx=0x7f82cc0b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.766+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82d40518e0 con 0x7f82dc0ceed0 2023-10-16T08:17:18.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.926+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82dc074010 con 0x7f82dc0ceed0 2023-10-16T08:17:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.926+0000 7f82d2ffd640 1 -- 172.21.15.32:0/3511818598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f82d4054f00 con 0x7f82dc0ceed0 2023-10-16T08:17:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:18.933 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f82bc0664d0 msgr2=0x7f82bc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f82bc0664d0 0x7f82bc068990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0aa000 tx=0x7f82cc0b0040 comp rx=0 tx=0).stop 2023-10-16T08:17:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 msgr2=0x7f82dc0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 0x7f82dc0cf350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82d4011c80 tx=0x7f82d400f580 comp rx=0 tx=0).stop 2023-10-16T08:17:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 shutdown_connections 2023-10-16T08:17:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82dc0ceed0 0x7f82dc0cf350 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f82bc0664d0 0x7f82bc068990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82dc0b6510 0x7f82dc150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 --2- 172.21.15.32:0/3511818598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c8003980 0x7f82dc150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 >> 172.21.15.32:0/3511818598 conn(0x7f82dc09bc50 msgr2=0x7f82dc0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 shutdown_connections 2023-10-16T08:17:18.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:18.930+0000 7f82e0c17640 1 -- 172.21.15.32:0/3511818598 wait complete. 2023-10-16T08:17:18.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:18.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:19.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 Processor -- start 2023-10-16T08:17:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 -- start start 2023-10-16T08:17:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faad80ceda0 0x7faad80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad80cc330 con 0x7faad8161260 2023-10-16T08:17:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad80cb720 con 0x7faad80ceda0 2023-10-16T08:17:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadf165640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad80cba00 con 0x7faad815cc60 2023-10-16T08:17:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadd702640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faad80ceda0 0x7faad80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadd702640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faad80ceda0 0x7faad80cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43178/0 (socket says 172.21.15.32:43178) 2023-10-16T08:17:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadd702640 1 -- 172.21.15.32:0/261251383 learned_addr learned my addr 172.21.15.32:0/261251383 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadcf01640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadd702640 1 -- 172.21.15.32:0/261251383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 msgr2=0x7faad80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadd702640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.098+0000 7faadd702640 1 -- 172.21.15.32:0/261251383 >> v1:172.21.15.186:6789/0 conn(0x7faad8161260 legacy=0x7faad80ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadd702640 1 -- 172.21.15.32:0/261251383 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad80cbce0 con 0x7faad80ceda0 2023-10-16T08:17:19.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadcf01640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadd702640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad80ceda0 0x7faad80cc990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faad0009e50 tx=0x7faad001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e7af2e05a0961f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faad77fe640 1 -- 172.21.15.32:0/261251383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00a58d0 con 0x7faad80ceda0 2023-10-16T08:17:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faad77fe640 1 -- 172.21.15.32:0/261251383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad00a5a30 con 0x7faad80ceda0 2023-10-16T08:17:19.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faad77fe640 1 -- 172.21.15.32:0/261251383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00a5dc0 con 0x7faad80ceda0 2023-10-16T08:17:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 -- 172.21.15.32:0/261251383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad80ceda0 msgr2=0x7faad80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad80ceda0 0x7faad80cc990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faad0009e50 tx=0x7faad001c9e0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 -- 172.21.15.32:0/261251383 shutdown_connections 2023-10-16T08:17:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad80ceda0 0x7faad80cc990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 --2- 172.21.15.32:0/261251383 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 -- 172.21.15.32:0/261251383 >> 172.21.15.32:0/261251383 conn(0x7faad806e570 msgr2=0x7faad806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 -- 172.21.15.32:0/261251383 shutdown_connections 2023-10-16T08:17:19.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.102+0000 7faadf165640 1 -- 172.21.15.32:0/261251383 wait complete. 2023-10-16T08:17:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 Processor -- start 2023-10-16T08:17:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 -- start start 2023-10-16T08:17:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad80ceda0 0x7faad8152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 0x7faad80b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad815a770 con 0x7faad8161260 2023-10-16T08:17:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8159b60 con 0x7faad80ceda0 2023-10-16T08:17:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadf165640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad8159e40 con 0x7faad815cc60 2023-10-16T08:17:19.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 0x7faad80b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadcf01640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad80ceda0 0x7faad8152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 0x7faad80b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43190/0 (socket says 172.21.15.32:43190) 2023-10-16T08:17:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faad7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadcf01640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad80ceda0 0x7faad8152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:47468/0 (socket says 172.21.15.32:47468) 2023-10-16T08:17:19.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 -- 172.21.15.32:0/1867365471 learned_addr learned my addr 172.21.15.32:0/1867365471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 -- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 msgr2=0x7faad80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 -- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad80ceda0 msgr2=0x7faad8152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad80ceda0 0x7faad8152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.106+0000 7faadd702640 1 -- 172.21.15.32:0/1867365471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad0009b30 con 0x7faad8161260 2023-10-16T08:17:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faadd702640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 0x7faad80b46c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faad00af040 tx=0x7faad0019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00b8670 con 0x7faad8161260 2023-10-16T08:17:19.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad00b87d0 con 0x7faad8161260 2023-10-16T08:17:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad815a120 con 0x7faad8161260 2023-10-16T08:17:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad00b8b80 con 0x7faad8161260 2023-10-16T08:17:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81525b0 con 0x7faad8161260 2023-10-16T08:17:19.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad815a8e0 con 0x7faad8161260 2023-10-16T08:17:19.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faad00b8ce0 con 0x7faad8161260 2023-10-16T08:17:19.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faad57fa640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faabc0664a0 0x7faabc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faad011f840 con 0x7faad8161260 2023-10-16T08:17:19.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.110+0000 7faadcf01640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faabc0664a0 0x7faabc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.114+0000 7faadcf01640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faabc0664a0 0x7faabc068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faacc009950 tx=0x7faacc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.114+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faad00ec9a0 con 0x7faad8161260 2023-10-16T08:17:19.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faad804e170 con 0x7faad8161260 2023-10-16T08:17:19.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faad57fa640 1 -- 172.21.15.32:0/1867365471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7faad00effc0 con 0x7faad8161260 2023-10-16T08:17:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:19.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faabc0664a0 msgr2=0x7faabc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faabc0664a0 0x7faabc068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faacc009950 tx=0x7faacc008040 comp rx=0 tx=0).stop 2023-10-16T08:17:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 msgr2=0x7faad80b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 0x7faad80b46c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faad00af040 tx=0x7faad0019780 comp rx=0 tx=0).stop 2023-10-16T08:17:19.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 shutdown_connections 2023-10-16T08:17:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faad8161260 0x7faad80b46c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faabc0664a0 0x7faabc068960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faad815cc60 0x7faad80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 --2- 172.21.15.32:0/1867365471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faad80ceda0 0x7faad8152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 >> 172.21.15.32:0/1867365471 conn(0x7faad806e570 msgr2=0x7faad806b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:19.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 shutdown_connections 2023-10-16T08:17:19.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.274+0000 7faadf165640 1 -- 172.21.15.32:0/1867365471 wait complete. 2023-10-16T08:17:19.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 Processor -- start 2023-10-16T08:17:19.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 -- start start 2023-10-16T08:17:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28f806d010 0x7f28f80c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f80c26b0 0x7f28f80c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f815ae30 con 0x7f28f8069e90 2023-10-16T08:17:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80cb090 con 0x7f28f806d010 2023-10-16T08:17:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28ff380640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f806d470 con 0x7f28f80c26b0 2023-10-16T08:17:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28f806d010 0x7f28f80c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28f806d010 0x7f28f80c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43194/0 (socket says 172.21.15.32:43194) 2023-10-16T08:17:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 -- 172.21.15.32:0/435228651 learned_addr learned my addr 172.21.15.32:0/435228651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fd91d640 1 --2- 172.21.15.32:0/435228651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f80c26b0 0x7f28f80c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 -- 172.21.15.32:0/435228651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f80c26b0 msgr2=0x7f28f80c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 --2- 172.21.15.32:0/435228651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f80c26b0 0x7f28f80c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 -- 172.21.15.32:0/435228651 >> v1:172.21.15.186:6789/0 conn(0x7f28f8069e90 legacy=0x7f28f806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.442+0000 7f28fc91b640 1 -- 172.21.15.32:0/435228651 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80ca510 con 0x7f28f806d010 2023-10-16T08:17:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28fc91b640 1 --2- 172.21.15.32:0/435228651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c9ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009870 tx=0x7f28e801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d60cbdfc650579a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28effff640 1 -- 172.21.15.32:0/435228651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a8a30 con 0x7f28f806d010 2023-10-16T08:17:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28effff640 1 -- 172.21.15.32:0/435228651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80a8b90 con 0x7f28f806d010 2023-10-16T08:17:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28effff640 1 -- 172.21.15.32:0/435228651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80b0440 con 0x7f28f806d010 2023-10-16T08:17:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 -- 172.21.15.32:0/435228651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 msgr2=0x7f28f80c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 --2- 172.21.15.32:0/435228651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c9ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009870 tx=0x7f28e801ed30 comp rx=0 tx=0).stop 2023-10-16T08:17:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 -- 172.21.15.32:0/435228651 shutdown_connections 2023-10-16T08:17:19.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 --2- 172.21.15.32:0/435228651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f80c26b0 0x7f28f80c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 --2- 172.21.15.32:0/435228651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c9ce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 -- 172.21.15.32:0/435228651 >> 172.21.15.32:0/435228651 conn(0x7f28f80b0da0 msgr2=0x7f28f80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 -- 172.21.15.32:0/435228651 shutdown_connections 2023-10-16T08:17:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.446+0000 7f28ff380640 1 -- 172.21.15.32:0/435228651 wait complete. 2023-10-16T08:17:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 Processor -- start 2023-10-16T08:17:19.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- start start 2023-10-16T08:17:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8069e90 0x7f28f8149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f80c26b0 0x7f28f80ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c8990 con 0x7f28f806d010 2023-10-16T08:17:19.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c7650 con 0x7f28f80c26b0 2023-10-16T08:17:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c78f0 con 0x7f28f8069e90 2023-10-16T08:17:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43210/0 (socket says 172.21.15.32:43210) 2023-10-16T08:17:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 -- 172.21.15.32:0/3197367273 learned_addr learned my addr 172.21.15.32:0/3197367273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:19.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fd11c640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8069e90 0x7f28f8149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fd91d640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f80c26b0 0x7f28f80ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 -- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8069e90 msgr2=0x7f28f8149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8069e90 0x7f28f8149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 -- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f80c26b0 msgr2=0x7f28f80ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f80c26b0 0x7f28f80ac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 -- 172.21.15.32:0/3197367273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80094d0 con 0x7f28f806d010 2023-10-16T08:17:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28fc91b640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c5ac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f28e801c960 tx=0x7f28e801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80be070 con 0x7f28f806d010 2023-10-16T08:17:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80ba400 con 0x7f28f806d010 2023-10-16T08:17:19.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80ba7b0 con 0x7f28f806d010 2023-10-16T08:17:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80c7bd0 con 0x7f28f806d010 2023-10-16T08:17:19.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f80c8190 con 0x7f28f806d010 2023-10-16T08:17:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.450+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f80cad90 con 0x7f28f806d010 2023-10-16T08:17:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.454+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f28e80ba910 con 0x7f28f806d010 2023-10-16T08:17:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.454+0000 7f28edffb640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28dc0664a0 0x7f28dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.454+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f28e8121fe0 con 0x7f28f806d010 2023-10-16T08:17:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.454+0000 7f28fd11c640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28dc0664a0 0x7f28dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.454+0000 7f28fd11c640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28dc0664a0 0x7f28dc068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000f70 tx=0x7f28f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.458+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f28f80cad90 con 0x7f28f806d010 2023-10-16T08:17:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28f80cad90 con 0x7f28f806d010 2023-10-16T08:17:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28edffb640 1 -- 172.21.15.32:0/3197367273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28f80cad90 con 0x7f28f806d010 2023-10-16T08:17:19.619 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:19.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28dc0664a0 msgr2=0x7f28dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28dc0664a0 0x7f28dc068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000f70 tx=0x7f28f4009040 comp rx=0 tx=0).stop 2023-10-16T08:17:19.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 msgr2=0x7f28f80c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c5ac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f28e801c960 tx=0x7f28e801bad0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 shutdown_connections 2023-10-16T08:17:19.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28f80c26b0 0x7f28f80ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28f806d010 0x7f28f80c5ac0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28dc0664a0 0x7f28dc068960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 --2- 172.21.15.32:0/3197367273 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f28f8069e90 0x7f28f8149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 >> 172.21.15.32:0/3197367273 conn(0x7f28f80b0da0 msgr2=0x7f28f80ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:19.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 shutdown_connections 2023-10-16T08:17:19.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.614+0000 7f28ff380640 1 -- 172.21.15.32:0/3197367273 wait complete. 2023-10-16T08:17:19.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.778+0000 7f10f67f8640 1 Processor -- start 2023-10-16T08:17:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.778+0000 7f10f67f8640 1 -- start start 2023-10-16T08:17:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10f67f8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10f0159900 0x7f10f006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10f67f8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f10f00b3f00 0x7f10f0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10f67f8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00b4390 con 0x7f10f006a3d0 2023-10-16T08:17:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10f67f8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00cd740 con 0x7f10f0159900 2023-10-16T08:17:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10f67f8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f006a840 con 0x7f10f00b3f00 2023-10-16T08:17:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10f0159900 0x7f10f006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10f0159900 0x7f10f006e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43222/0 (socket says 172.21.15.32:43222) 2023-10-16T08:17:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 -- 172.21.15.32:0/2689758128 learned_addr learned my addr 172.21.15.32:0/2689758128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 -- 172.21.15.32:0/2689758128 >> v2:172.21.15.186:3300/0 conn(0x7f10f00b3f00 msgr2=0x7f10f0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 --2- 172.21.15.32:0/2689758128 >> v2:172.21.15.186:3300/0 conn(0x7f10f00b3f00 0x7f10f0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 -- 172.21.15.32:0/2689758128 >> v1:172.21.15.186:6789/0 conn(0x7f10f006a3d0 legacy=0x7f10f00cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 -- 172.21.15.32:0/2689758128 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00b4600 con 0x7f10f0159900 2023-10-16T08:17:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10effff640 1 --2- 172.21.15.32:0/2689758128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f006e260 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10f0071620 tx=0x7f10e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73580f90e76d7179 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.782+0000 7f10eeffd640 1 -- 172.21.15.32:0/2689758128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00aa5f0 con 0x7f10f0159900 2023-10-16T08:17:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10eeffd640 1 -- 172.21.15.32:0/2689758128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e00aa750 con 0x7f10f0159900 2023-10-16T08:17:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10eeffd640 1 -- 172.21.15.32:0/2689758128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00aaae0 con 0x7f10f0159900 2023-10-16T08:17:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- 172.21.15.32:0/2689758128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 msgr2=0x7f10f006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 --2- 172.21.15.32:0/2689758128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f006e260 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10f0071620 tx=0x7f10e001c490 comp rx=0 tx=0).stop 2023-10-16T08:17:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- 172.21.15.32:0/2689758128 shutdown_connections 2023-10-16T08:17:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 --2- 172.21.15.32:0/2689758128 >> v2:172.21.15.186:3300/0 conn(0x7f10f00b3f00 0x7f10f0069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 --2- 172.21.15.32:0/2689758128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f006e260 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- 172.21.15.32:0/2689758128 >> 172.21.15.32:0/2689758128 conn(0x7f10f00b0da0 msgr2=0x7f10f00ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- 172.21.15.32:0/2689758128 shutdown_connections 2023-10-16T08:17:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- 172.21.15.32:0/2689758128 wait complete. 2023-10-16T08:17:19.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 Processor -- start 2023-10-16T08:17:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- start start 2023-10-16T08:17:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f006a3d0 0x7f10f00c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f00b3f00 0x7f10f014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f014d230 con 0x7f10f0159900 2023-10-16T08:17:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00c2740 con 0x7f10f006a3d0 2023-10-16T08:17:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f67f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00c2a20 con 0x7f10f00b3f00 2023-10-16T08:17:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f4d95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f4d95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f014fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43226/0 (socket says 172.21.15.32:43226) 2023-10-16T08:17:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10f4d95640 1 -- 172.21.15.32:0/1384938337 learned_addr learned my addr 172.21.15.32:0/1384938337 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10effff640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f006a3d0 0x7f10f00c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.786+0000 7f10ef7fe640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f00b3f00 0x7f10f014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f4d95640 1 -- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f00b3f00 msgr2=0x7f10f014ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f4d95640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f00b3f00 0x7f10f014ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f4d95640 1 -- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f006a3d0 msgr2=0x7f10f00c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f4d95640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f006a3d0 0x7f10f00c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f4d95640 1 -- 172.21.15.32:0/1384938337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00094d0 con 0x7f10f0159900 2023-10-16T08:17:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f4d95640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f014fb10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c780 tx=0x7f10e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8003d90 con 0x7f10f0159900 2023-10-16T08:17:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e800ce90 con 0x7f10f0159900 2023-10-16T08:17:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00c2d00 con 0x7f10f0159900 2023-10-16T08:17:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80052b0 con 0x7f10f0159900 2023-10-16T08:17:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f00c32c0 con 0x7f10f0159900 2023-10-16T08:17:19.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f00c5150 con 0x7f10f0159900 2023-10-16T08:17:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10e8005410 con 0x7f10f0159900 2023-10-16T08:17:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10ecff9640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10dc0664a0 0x7f10dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f10e8083a40 con 0x7f10f0159900 2023-10-16T08:17:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.790+0000 7f10effff640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10dc0664a0 0x7f10dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.794+0000 7f10effff640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10dc0664a0 0x7f10dc068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10e00ae000 tx=0x7f10e00b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.794+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10e8050ba0 con 0x7f10f0159900 2023-10-16T08:17:19.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.954+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10f00c34e0 con 0x7f10f0159900 2023-10-16T08:17:19.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.954+0000 7f10ecff9640 1 -- 172.21.15.32:0/1384938337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10e8017020 con 0x7f10f0159900 2023-10-16T08:17:19.961 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:19.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10dc0664a0 msgr2=0x7f10dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10dc0664a0 0x7f10dc068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10e00ae000 tx=0x7f10e00b5040 comp rx=0 tx=0).stop 2023-10-16T08:17:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 msgr2=0x7f10f014fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f014fb10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c780 tx=0x7f10e800cc40 comp rx=0 tx=0).stop 2023-10-16T08:17:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 shutdown_connections 2023-10-16T08:17:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0159900 0x7f10f014fb10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f10dc0664a0 0x7f10dc068960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f10f00b3f00 0x7f10f014ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 --2- 172.21.15.32:0/1384938337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10f006a3d0 0x7f10f00c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 >> 172.21.15.32:0/1384938337 conn(0x7f10f00b0da0 msgr2=0x7f10f00701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 shutdown_connections 2023-10-16T08:17:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:19.958+0000 7f10f67f8640 1 -- 172.21.15.32:0/1384938337 wait complete. 2023-10-16T08:17:19.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2719d640 1 Processor -- start 2023-10-16T08:17:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2719d640 1 -- start start 2023-10-16T08:17:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2719d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5b200d0210 0x7f5b200d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2719d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200d1670 con 0x7f5b200b84e0 2023-10-16T08:17:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2719d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200d1950 con 0x7f5b2015cba0 2023-10-16T08:17:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2719d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b200d1460 con 0x7f5b200d0210 2023-10-16T08:17:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2573a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b200b84e0 0x7f5b200b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45474/0 (socket says 172.21.15.32:45474) 2023-10-16T08:17:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b2573a640 1 -- 172.21.15.32:0/3060108430 learned_addr learned my addr 172.21.15.32:0/3060108430 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1ffff640 1 --2- 172.21.15.32:0/3060108430 >> v2:172.21.15.186:3300/0 conn(0x7f5b200d0210 0x7f5b200d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743494968 0 0) 0x7f5b200d1670 con 0x7f5b200b84e0 2023-10-16T08:17:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b10003530 con 0x7f5b200b84e0 2023-10-16T08:17:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296724291 0 0) 0x7f5b200d1950 con 0x7f5b2015cba0 2023-10-16T08:17:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b200d1670 con 0x7f5b2015cba0 2023-10-16T08:17:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1ffff640 1 -- 172.21.15.32:0/3060108430 >> v1:172.21.15.186:6789/0 conn(0x7f5b2015cba0 legacy=0x7f5b200cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1ffff640 1 -- 172.21.15.32:0/3060108430 >> v1:172.21.15.32:6789/0 conn(0x7f5b200b84e0 legacy=0x7f5b200b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.126+0000 7f5b1ffff640 1 -- 172.21.15.32:0/3060108430 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200d1c30 con 0x7f5b200d0210 2023-10-16T08:17:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b1ffff640 1 --2- 172.21.15.32:0/3060108430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200d0f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0097c0 tx=0x7f5b0c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba1f9f7e7bc4a22c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0ab5f0 con 0x7f5b200d0210 2023-10-16T08:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0ab750 con 0x7f5b200d0210 2023-10-16T08:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b1f7fe640 1 -- 172.21.15.32:0/3060108430 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0abae0 con 0x7f5b200d0210 2023-10-16T08:17:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 -- 172.21.15.32:0/3060108430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 msgr2=0x7f5b200d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 --2- 172.21.15.32:0/3060108430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200d0f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0097c0 tx=0x7f5b0c01c3f0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 -- 172.21.15.32:0/3060108430 shutdown_connections 2023-10-16T08:17:20.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 --2- 172.21.15.32:0/3060108430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200d0f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 -- 172.21.15.32:0/3060108430 >> 172.21.15.32:0/3060108430 conn(0x7f5b200719c0 msgr2=0x7f5b20071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:20.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 -- 172.21.15.32:0/3060108430 shutdown_connections 2023-10-16T08:17:20.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.130+0000 7f5b2719d640 1 -- 172.21.15.32:0/3060108430 wait complete. 2023-10-16T08:17:20.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 Processor -- start 2023-10-16T08:17:20.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 -- start start 2023-10-16T08:17:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 0x7f5b20150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b2015cba0 0x7f5b20158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200c6e90 con 0x7f5b200b84e0 2023-10-16T08:17:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200c6280 con 0x7f5b200d0210 2023-10-16T08:17:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200c7ce0 con 0x7f5b2015cba0 2023-10-16T08:17:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 0x7f5b20150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 0x7f5b20150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43236/0 (socket says 172.21.15.32:43236) 2023-10-16T08:17:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 -- 172.21.15.32:0/1386918098 learned_addr learned my addr 172.21.15.32:0/1386918098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b1ffff640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2573a640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b2015cba0 0x7f5b20158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 -- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b2015cba0 msgr2=0x7f5b20158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b2015cba0 0x7f5b20158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 -- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 msgr2=0x7f5b200c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 -- 172.21.15.32:0/1386918098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c009420 con 0x7f5b200b84e0 2023-10-16T08:17:20.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b24f39640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 0x7f5b20150130 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c460 tx=0x7f5b1400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b140052d0 con 0x7f5b200b84e0 2023-10-16T08:17:20.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14005450 con 0x7f5b200b84e0 2023-10-16T08:17:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b200c64f0 con 0x7f5b200b84e0 2023-10-16T08:17:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14012870 con 0x7f5b200b84e0 2023-10-16T08:17:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.134+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b200c6ab0 con 0x7f5b200b84e0 2023-10-16T08:17:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.138+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20150670 con 0x7f5b200b84e0 2023-10-16T08:17:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.138+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5b140129d0 con 0x7f5b200b84e0 2023-10-16T08:17:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.138+0000 7f5b1d7fa640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5b040664a0 0x7f5b04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.138+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5b14083d50 con 0x7f5b200b84e0 2023-10-16T08:17:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.138+0000 7f5b1ffff640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5b040664a0 0x7f5b04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.138+0000 7f5b1ffff640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5b040664a0 0x7f5b04068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0af000 tx=0x7f5b0c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.142+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5b14050d90 con 0x7f5b200b84e0 2023-10-16T08:17:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.302+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b2004e170 con 0x7f5b200b84e0 2023-10-16T08:17:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.302+0000 7f5b1d7fa640 1 -- 172.21.15.32:0/1386918098 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b140543b0 con 0x7f5b200b84e0 2023-10-16T08:17:20.308 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.302+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5b040664a0 msgr2=0x7f5b04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.302+0000 7f5b2719d640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5b040664a0 0x7f5b04068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0af000 tx=0x7f5b0c0a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.302+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 msgr2=0x7f5b20150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.302+0000 7f5b2719d640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 0x7f5b20150130 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c460 tx=0x7f5b1400c920 comp rx=0 tx=0).stop 2023-10-16T08:17:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 shutdown_connections 2023-10-16T08:17:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5b040664a0 0x7f5b04068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5b2015cba0 0x7f5b20158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b200d0210 0x7f5b200c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 --2- 172.21.15.32:0/1386918098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b200b84e0 0x7f5b20150130 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 >> 172.21.15.32:0/1386918098 conn(0x7f5b200719c0 msgr2=0x7f5b2006b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 shutdown_connections 2023-10-16T08:17:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.306+0000 7f5b2719d640 1 -- 172.21.15.32:0/1386918098 wait complete. 2023-10-16T08:17:20.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:20.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 Processor -- start 2023-10-16T08:17:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 -- start start 2023-10-16T08:17:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89015ca70 0x7ff8900d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8900d3550 0x7ff8900cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8900c65f0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900ce820 con 0x7ff8900d3550 2023-10-16T08:17:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff8974a9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900cfa40 con 0x7ff89015ca70 2023-10-16T08:17:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8900d3550 0x7ff8900cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8900d3550 0x7ff8900cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43250/0 (socket says 172.21.15.32:43250) 2023-10-16T08:17:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 -- 172.21.15.32:0/859948280 learned_addr learned my addr 172.21.15.32:0/859948280 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff895a46640 1 --1- 172.21.15.32:0/859948280 >> v1:172.21.15.32:6789/0 conn(0x7ff8900cf5d0 0x7ff8900d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45490/0 (socket says 172.21.15.32:45490) 2023-10-16T08:17:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff895245640 1 --2- 172.21.15.32:0/859948280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89015ca70 0x7ff8900d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 -- 172.21.15.32:0/859948280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89015ca70 msgr2=0x7ff8900d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 --2- 172.21.15.32:0/859948280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89015ca70 0x7ff8900d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 -- 172.21.15.32:0/859948280 >> v1:172.21.15.32:6789/0 conn(0x7ff8900cf5d0 legacy=0x7ff8900d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff894a44640 1 -- 172.21.15.32:0/859948280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900cdf80 con 0x7ff8900d3550 2023-10-16T08:17:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.470+0000 7ff887fff640 1 -- 172.21.15.32:0/859948280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793223345 0 0) 0x7ff8900c65f0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff894a44640 1 --2- 172.21.15.32:0/859948280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900d3550 0x7ff8900cf090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8800098a0 tx=0x7ff88001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6080c9e4cc8bf4c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff887fff640 1 -- 172.21.15.32:0/859948280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8800aa650 con 0x7ff8900d3550 2023-10-16T08:17:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff887fff640 1 -- 172.21.15.32:0/859948280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8800aa7b0 con 0x7ff8900d3550 2023-10-16T08:17:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff887fff640 1 -- 172.21.15.32:0/859948280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8800aab40 con 0x7ff8900d3550 2023-10-16T08:17:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 -- 172.21.15.32:0/859948280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900d3550 msgr2=0x7ff8900cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 --2- 172.21.15.32:0/859948280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900d3550 0x7ff8900cf090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8800098a0 tx=0x7ff88001c4f0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 -- 172.21.15.32:0/859948280 shutdown_connections 2023-10-16T08:17:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 --2- 172.21.15.32:0/859948280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900d3550 0x7ff8900cf090 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 --2- 172.21.15.32:0/859948280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff89015ca70 0x7ff8900d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 -- 172.21.15.32:0/859948280 >> 172.21.15.32:0/859948280 conn(0x7ff89009bc50 msgr2=0x7ff8900d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 -- 172.21.15.32:0/859948280 shutdown_connections 2023-10-16T08:17:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.474+0000 7ff8974a9640 1 -- 172.21.15.32:0/859948280 wait complete. 2023-10-16T08:17:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 Processor -- start 2023-10-16T08:17:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 -- start start 2023-10-16T08:17:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 0x7ff89014c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8900d3550 0x7ff890153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff89015ca70 0x7ff890153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89014ce00 con 0x7ff8900cf5d0 2023-10-16T08:17:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890157530 con 0x7ff89015ca70 2023-10-16T08:17:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff890157810 con 0x7ff8900d3550 2023-10-16T08:17:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 0x7ff89014c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 0x7ff89014c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43262/0 (socket says 172.21.15.32:43262) 2023-10-16T08:17:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 -- 172.21.15.32:0/1389062684 learned_addr learned my addr 172.21.15.32:0/1389062684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff894a44640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8900d3550 0x7ff890153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895a46640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff89015ca70 0x7ff890153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 -- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8900d3550 msgr2=0x7ff890153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8900d3550 0x7ff890153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 -- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff89015ca70 msgr2=0x7ff890153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff89015ca70 0x7ff890153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 -- 172.21.15.32:0/1389062684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800094d0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff895245640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 0x7ff89014c8c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00e870 tx=0x7ff88c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0095d0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c00c940 con 0x7ff8900cf5d0 2023-10-16T08:17:20.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff890157af0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00ccb0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.478+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8901580b0 con 0x7ff8900cf5d0 2023-10-16T08:17:20.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.482+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8900b3820 con 0x7ff8900cf5d0 2023-10-16T08:17:20.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.482+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff88c013470 con 0x7ff8900cf5d0 2023-10-16T08:17:20.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.482+0000 7ff885ffb640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff8740664a0 0x7ff874068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.482+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff88c084800 con 0x7ff8900cf5d0 2023-10-16T08:17:20.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.482+0000 7ff894a44640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff8740664a0 0x7ff874068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.482+0000 7ff894a44640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff8740664a0 0x7ff874068960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8800ae000 tx=0x7ff8800a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.486+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff88c051970 con 0x7ff8900cf5d0 2023-10-16T08:17:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff890074010 con 0x7ff8900cf5d0 2023-10-16T08:17:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff885ffb640 1 -- 172.21.15.32:0/1389062684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff88c054f90 con 0x7ff8900cf5d0 2023-10-16T08:17:20.660 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff8740664a0 msgr2=0x7ff874068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff8740664a0 0x7ff874068960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8800ae000 tx=0x7ff8800a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 msgr2=0x7ff89014c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 0x7ff89014c8c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00e870 tx=0x7ff88c00ed30 comp rx=0 tx=0).stop 2023-10-16T08:17:20.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 shutdown_connections 2023-10-16T08:17:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff89015ca70 0x7ff890153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff8740664a0 0x7ff874068960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff8900d3550 0x7ff890153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 --2- 172.21.15.32:0/1389062684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8900cf5d0 0x7ff89014c8c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 >> 172.21.15.32:0/1389062684 conn(0x7ff89009bc50 msgr2=0x7ff8900d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 shutdown_connections 2023-10-16T08:17:20.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.654+0000 7ff8974a9640 1 -- 172.21.15.32:0/1389062684 wait complete. 2023-10-16T08:17:20.681 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:17:20.681 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:20.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb43d9640 1 Processor -- start 2023-10-16T08:17:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb43d9640 1 -- start start 2023-10-16T08:17:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb43d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbeac15a880 0x7fbeac15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb43d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac1614a0 con 0x7fbeac0b00e0 2023-10-16T08:17:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb43d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac15e000 con 0x7fbeac0cda30 2023-10-16T08:17:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb43d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac0cb1d0 con 0x7fbeac15a880 2023-10-16T08:17:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb1974640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbeac0b00e0 0x7fbeac160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45494/0 (socket says 172.21.15.32:45494) 2023-10-16T08:17:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.870+0000 7fbeb1974640 1 -- 172.21.15.32:0/2577141487 learned_addr learned my addr 172.21.15.32:0/2577141487 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb2175640 1 --2- 172.21.15.32:0/2577141487 >> v2:172.21.15.186:3300/0 conn(0x7fbeac15a880 0x7fbeac15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb1173640 1 -- 172.21.15.32:0/2577141487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818260460 0 0) 0x7fbeac1614a0 con 0x7fbeac0b00e0 2023-10-16T08:17:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb1173640 1 -- 172.21.15.32:0/2577141487 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe98003530 con 0x7fbeac0b00e0 2023-10-16T08:17:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb1173640 1 -- 172.21.15.32:0/2577141487 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514647469 0 0) 0x7fbe98003530 con 0x7fbeac0b00e0 2023-10-16T08:17:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb2175640 1 -- 172.21.15.32:0/2577141487 >> v1:172.21.15.186:6789/0 conn(0x7fbeac0cda30 legacy=0x7fbeac160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb2175640 1 -- 172.21.15.32:0/2577141487 >> v1:172.21.15.32:6789/0 conn(0x7fbeac0b00e0 legacy=0x7fbeac160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb2175640 1 -- 172.21.15.32:0/2577141487 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0cb4b0 con 0x7fbeac15a880 2023-10-16T08:17:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb2175640 1 --2- 172.21.15.32:0/2577141487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac15a880 0x7fbeac15dac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbea80097c0 tx=0x7fbea801c250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e058351b7cafe30f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.874+0000 7fbeb1173640 1 -- 172.21.15.32:0/2577141487 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c001680 con 0x7fbeac15a880 2023-10-16T08:17:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb1173640 1 -- 172.21.15.32:0/2577141487 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea80a7050 con 0x7fbeac15a880 2023-10-16T08:17:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb1173640 1 -- 172.21.15.32:0/2577141487 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea80ab620 con 0x7fbeac15a880 2023-10-16T08:17:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2577141487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac15a880 msgr2=0x7fbeac15dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2577141487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac15a880 0x7fbeac15dac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbea80097c0 tx=0x7fbea801c250 comp rx=0 tx=0).stop 2023-10-16T08:17:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2577141487 shutdown_connections 2023-10-16T08:17:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2577141487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac15a880 0x7fbeac15dac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2577141487 >> 172.21.15.32:0/2577141487 conn(0x7fbeac06c190 msgr2=0x7fbeac06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:20.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2577141487 shutdown_connections 2023-10-16T08:17:20.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2577141487 wait complete. 2023-10-16T08:17:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 Processor -- start 2023-10-16T08:17:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- start start 2023-10-16T08:17:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac0b00e0 0x7fbeac1f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 0x7fbeac1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeac15a880 0x7fbeac159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1f6670 con 0x7fbeac0cda30 2023-10-16T08:17:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0c6cc0 con 0x7fbeac0b00e0 2023-10-16T08:17:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb43d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0c6140 con 0x7fbeac15a880 2023-10-16T08:17:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb1974640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 0x7fbeac1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb1974640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 0x7fbeac1f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43276/0 (socket says 172.21.15.32:43276) 2023-10-16T08:17:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb1974640 1 -- 172.21.15.32:0/2127165227 learned_addr learned my addr 172.21.15.32:0/2127165227 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb2175640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac0b00e0 0x7fbeac1f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.878+0000 7fbeb2976640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeac15a880 0x7fbeac159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb1974640 1 -- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeac15a880 msgr2=0x7fbeac159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb1974640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeac15a880 0x7fbeac159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb1974640 1 -- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac0b00e0 msgr2=0x7fbeac1f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb1974640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac0b00e0 0x7fbeac1f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb1974640 1 -- 172.21.15.32:0/2127165227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8009420 con 0x7fbeac0cda30 2023-10-16T08:17:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb1974640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 0x7fbeac1f6130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007fd0 tx=0x7fbe9c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0072d0 con 0x7fbeac0cda30 2023-10-16T08:17:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c007450 con 0x7fbeac0cda30 2023-10-16T08:17:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac0c6420 con 0x7fbeac0cda30 2023-10-16T08:17:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0047f0 con 0x7fbeac0cda30 2023-10-16T08:17:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac0c69e0 con 0x7fbeac0cda30 2023-10-16T08:17:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac074010 con 0x7fbeac0cda30 2023-10-16T08:17:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe9c004950 con 0x7fbeac0cda30 2023-10-16T08:17:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbea2ffd640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe900664a0 0x7fbe90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbe9c083800 con 0x7fbeac0cda30 2023-10-16T08:17:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.882+0000 7fbeb2175640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe900664a0 0x7fbe90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.886+0000 7fbeb2175640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe900664a0 0x7fbe90068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbea80ad000 tx=0x7fbea8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:20.886+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe9c050960 con 0x7fbeac0cda30 2023-10-16T08:17:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.046+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbeac0c7680 con 0x7fbeac0cda30 2023-10-16T08:17:21.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbea2ffd640 1 -- 172.21.15.32:0/2127165227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fbe9c053f80 con 0x7fbeac0cda30 2023-10-16T08:17:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:21.055 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe900664a0 msgr2=0x7fbe90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe900664a0 0x7fbe90068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbea80ad000 tx=0x7fbea8008040 comp rx=0 tx=0).stop 2023-10-16T08:17:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 msgr2=0x7fbeac1f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 0x7fbeac1f6130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007fd0 tx=0x7fbe9c00cbe0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 shutdown_connections 2023-10-16T08:17:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe900664a0 0x7fbe90068960 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbeac15a880 0x7fbeac159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeac0cda30 0x7fbeac1f6130 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 --2- 172.21.15.32:0/2127165227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbeac0b00e0 0x7fbeac1f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 >> 172.21.15.32:0/2127165227 conn(0x7fbeac06c190 msgr2=0x7fbeac0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 shutdown_connections 2023-10-16T08:17:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.050+0000 7fbeb43d9640 1 -- 172.21.15.32:0/2127165227 wait complete. 2023-10-16T08:17:21.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:21.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:21.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f532a32a640 1 Processor -- start 2023-10-16T08:17:21.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f532a32a640 1 -- start start 2023-10-16T08:17:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f532a32a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53240d0210 0x7f53240d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f532a32a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240d1670 con 0x7f532415cba0 2023-10-16T08:17:21.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f532a32a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240d1950 con 0x7f53240b84e0 2023-10-16T08:17:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f532a32a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53240d1460 con 0x7f53240d0210 2023-10-16T08:17:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5323fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f532415cba0 0x7f53240cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45502/0 (socket says 172.21.15.32:45502) 2023-10-16T08:17:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5323fff640 1 -- 172.21.15.32:0/797719780 learned_addr learned my addr 172.21.15.32:0/797719780 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f53237fe640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53240d0210 0x7f53240d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921802560 0 0) 0x7f53240d1670 con 0x7f532415cba0 2023-10-16T08:17:21.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c003530 con 0x7f532415cba0 2023-10-16T08:17:21.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055549627 0 0) 0x7f53240d1950 con 0x7f53240b84e0 2023-10-16T08:17:21.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53240d1670 con 0x7f53240b84e0 2023-10-16T08:17:21.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.218+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2368078808 0 0) 0x7f530c003530 con 0x7f532415cba0 2023-10-16T08:17:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53240d0210 msgr2=0x7f53240d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53240d0210 0x7f53240d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 >> v1:172.21.15.186:6789/0 conn(0x7f53240b84e0 legacy=0x7f53240b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240d1c30 con 0x7f532415cba0 2023-10-16T08:17:21.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f5314001680 con 0x7f532415cba0 2023-10-16T08:17:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 >> v1:172.21.15.32:6789/0 conn(0x7f532415cba0 legacy=0x7f53240cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c003b40 0x7f530c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530c0047d0 0x7f530c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c005190 0x7f530c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53240d1460 con 0x7f530c005190 2023-10-16T08:17:21.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00bf80 con 0x7f530c003b40 2023-10-16T08:17:21.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c140 con 0x7f530c0047d0 2023-10-16T08:17:21.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53237fe640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53240d0210 0x7f53240d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:21.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c005190 0x7f530c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5323fff640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c003b40 0x7f530c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53237fe640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530c0047d0 0x7f530c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 -- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530c0047d0 msgr2=0x7f530c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530c0047d0 0x7f530c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 -- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c003b40 msgr2=0x7f530c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c003b40 0x7f530c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 -- 172.21.15.32:0/797719780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c003530 con 0x7f530c005190 2023-10-16T08:17:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f53288c7640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c005190 0x7f530c00b990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f531c001fd0 tx=0x7f531c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70869171638e4c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.222+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314007ea0 con 0x7f530c005190 2023-10-16T08:17:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c003ba0 con 0x7f530c005190 2023-10-16T08:17:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f5322ffd640 1 -- 172.21.15.32:0/797719780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0050d0 con 0x7f530c005190 2023-10-16T08:17:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- 172.21.15.32:0/797719780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c005190 msgr2=0x7f530c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 --2- 172.21.15.32:0/797719780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c005190 0x7f530c00b990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f531c001fd0 tx=0x7f531c01c8e0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f53288c7640 1 -- 172.21.15.32:0/797719780 reap_dead start 2023-10-16T08:17:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- 172.21.15.32:0/797719780 shutdown_connections 2023-10-16T08:17:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- 172.21.15.32:0/797719780 >> 172.21.15.32:0/797719780 conn(0x7f53240719c0 msgr2=0x7f5324071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- 172.21.15.32:0/797719780 shutdown_connections 2023-10-16T08:17:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- 172.21.15.32:0/797719780 wait complete. 2023-10-16T08:17:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 Processor -- start 2023-10-16T08:17:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- start start 2023-10-16T08:17:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c005190 0x7f5324149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 0x7f53240c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532415ee80 0x7f532415f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532415f870 con 0x7f532415e020 2023-10-16T08:17:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532415db90 con 0x7f530c005190 2023-10-16T08:17:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f532a32a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532415cf80 con 0x7f532415ee80 2023-10-16T08:17:21.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f53237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 0x7f53240c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f53237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 0x7f53240c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43286/0 (socket says 172.21.15.32:43286) 2023-10-16T08:17:21.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.226+0000 7f53237fe640 1 -- 172.21.15.32:0/3506079959 learned_addr learned my addr 172.21.15.32:0/3506079959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:21.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5323fff640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c005190 0x7f5324149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53288c7640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532415ee80 0x7f532415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53237fe640 1 -- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532415ee80 msgr2=0x7f532415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53237fe640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532415ee80 0x7f532415f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53237fe640 1 -- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c005190 msgr2=0x7f5324149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53237fe640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c005190 0x7f5324149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53237fe640 1 -- 172.21.15.32:0/3506079959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532415d260 con 0x7f532415e020 2023-10-16T08:17:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53288c7640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532415ee80 0x7f532415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f53237fe640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 0x7f53240c8d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5318011c30 tx=0x7f531800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531800fcc0 con 0x7f532415e020 2023-10-16T08:17:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531800fe20 con 0x7f532415e020 2023-10-16T08:17:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318014640 con 0x7f532415e020 2023-10-16T08:17:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532415d540 con 0x7f532415e020 2023-10-16T08:17:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532414a3e0 con 0x7f532415e020 2023-10-16T08:17:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532404e170 con 0x7f532415e020 2023-10-16T08:17:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53180147a0 con 0x7f532415e020 2023-10-16T08:17:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5320ff9640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53100664d0 0x7f5310068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.230+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5318084c70 con 0x7f532415e020 2023-10-16T08:17:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.234+0000 7f5323fff640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53100664d0 0x7f5310068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.234+0000 7f5323fff640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53100664d0 0x7f5310068990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5314001060 tx=0x7f5314009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.234+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5318051c90 con 0x7f532415e020 2023-10-16T08:17:21.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f532415dcd0 con 0x7f532415e020 2023-10-16T08:17:21.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f5320ff9640 1 -- 172.21.15.32:0/3506079959 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53180220a0 con 0x7f532415e020 2023-10-16T08:17:21.399 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:21.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53100664d0 msgr2=0x7f5310068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53100664d0 0x7f5310068990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5314001060 tx=0x7f5314009040 comp rx=0 tx=0).stop 2023-10-16T08:17:21.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 msgr2=0x7f53240c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 0x7f53240c8d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5318011c30 tx=0x7f531800f610 comp rx=0 tx=0).stop 2023-10-16T08:17:21.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 shutdown_connections 2023-10-16T08:17:21.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53100664d0 0x7f5310068990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f532415ee80 0x7f532415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f532415e020 0x7f53240c8d00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 --2- 172.21.15.32:0/3506079959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530c005190 0x7f5324149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 >> 172.21.15.32:0/3506079959 conn(0x7f53240719c0 msgr2=0x7f53240b6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 shutdown_connections 2023-10-16T08:17:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.393+0000 7f532a32a640 1 -- 172.21.15.32:0/3506079959 wait complete. 2023-10-16T08:17:21.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.557+0000 7feaff4dc640 1 Processor -- start 2023-10-16T08:17:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.557+0000 7feaff4dc640 1 -- start start 2023-10-16T08:17:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feaff4dc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feaf80b3cb0 0x7feaf80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feaff4dc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf80d0880 con 0x7feaf80cf210 2023-10-16T08:17:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feaff4dc640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf80b1c00 con 0x7feaf815cdc0 2023-10-16T08:17:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feaff4dc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80b1ee0 con 0x7feaf80b3cb0 2023-10-16T08:17:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafe4da640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feaf80cf210 0x7feaf80cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45504/0 (socket says 172.21.15.32:45504) 2023-10-16T08:17:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafe4da640 1 -- 172.21.15.32:0/386507542 learned_addr learned my addr 172.21.15.32:0/386507542 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafecdb640 1 --2- 172.21.15.32:0/386507542 >> v2:172.21.15.186:3300/0 conn(0x7feaf80b3cb0 0x7feaf80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729852923 0 0) 0x7feaf80d0880 con 0x7feaf80cf210 2023-10-16T08:17:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae4003530 con 0x7feaf80cf210 2023-10-16T08:17:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831402797 0 0) 0x7feaf80b1c00 con 0x7feaf815cdc0 2023-10-16T08:17:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf80d0880 con 0x7feaf815cdc0 2023-10-16T08:17:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735286339 0 0) 0x7feae4003530 con 0x7feaf80cf210 2023-10-16T08:17:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 >> v2:172.21.15.186:3300/0 conn(0x7feaf80b3cb0 msgr2=0x7feaf80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 --2- 172.21.15.32:0/386507542 >> v2:172.21.15.186:3300/0 conn(0x7feaf80b3cb0 0x7feaf80cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 >> v1:172.21.15.186:6789/0 conn(0x7feaf815cdc0 legacy=0x7feaf80cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80b21c0 con 0x7feaf80cf210 2023-10-16T08:17:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafecdb640 1 --2- 172.21.15.32:0/386507542 >> v2:172.21.15.186:3300/0 conn(0x7feaf80b3cb0 0x7feaf80cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7feaf4001800 con 0x7feaf80cf210 2023-10-16T08:17:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 >> v1:172.21.15.32:6789/0 conn(0x7feaf80cf210 legacy=0x7feaf80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feae4003b10 0x7feae4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feae4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feae4005190 0x7feae400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80b1ee0 con 0x7feae40047d0 2023-10-16T08:17:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae400bfb0 con 0x7feae4003b10 2023-10-16T08:17:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae400c170 con 0x7feae4005190 2023-10-16T08:17:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafdcd9640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feae4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafe4da640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feae4003b10 0x7feae4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafecdb640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feae4005190 0x7feae400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafdcd9640 1 -- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feae4005190 msgr2=0x7feae400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafdcd9640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feae4005190 0x7feae400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafdcd9640 1 -- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feae4003b10 msgr2=0x7feae4003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafdcd9640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feae4003b10 0x7feae4003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafdcd9640 1 -- 172.21.15.32:0/386507542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4003530 con 0x7feae40047d0 2023-10-16T08:17:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.561+0000 7feafecdb640 1 -- 172.21.15.32:0/386507542 reap_dead start 2023-10-16T08:17:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feafdcd9640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feae4004c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feae80018d0 tx=0x7feae801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1918a7dedf07fedb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80052d0 con 0x7feae40047d0 2023-10-16T08:17:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8005450 con 0x7feae40047d0 2023-10-16T08:17:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feafd4d8640 1 -- 172.21.15.32:0/386507542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80a88f0 con 0x7feae40047d0 2023-10-16T08:17:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 -- 172.21.15.32:0/386507542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 msgr2=0x7feae4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feae4004c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feae80018d0 tx=0x7feae801ec90 comp rx=0 tx=0).stop 2023-10-16T08:17:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 -- 172.21.15.32:0/386507542 shutdown_connections 2023-10-16T08:17:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 --2- 172.21.15.32:0/386507542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feae4004c50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 -- 172.21.15.32:0/386507542 >> 172.21.15.32:0/386507542 conn(0x7feaf80d2800 msgr2=0x7feaf80d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 -- 172.21.15.32:0/386507542 shutdown_connections 2023-10-16T08:17:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.565+0000 7feaff4dc640 1 -- 172.21.15.32:0/386507542 wait complete. 2023-10-16T08:17:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 Processor -- start 2023-10-16T08:17:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- start start 2023-10-16T08:17:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feaf80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feaf80ca8e0 0x7feaf80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaf80cad60 0x7feaf80cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80ca4c0 con 0x7feae40047d0 2023-10-16T08:17:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80c98d0 con 0x7feaf80cad60 2023-10-16T08:17:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80cc250 con 0x7feaf80ca8e0 2023-10-16T08:17:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feaf80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feaf80b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43300/0 (socket says 172.21.15.32:43300) 2023-10-16T08:17:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 -- 172.21.15.32:0/4253845460 learned_addr learned my addr 172.21.15.32:0/4253845460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafdcd9640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feaf80ca8e0 0x7feaf80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 -- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feaf80ca8e0 msgr2=0x7feaf80cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feaf80ca8e0 0x7feaf80cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 -- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaf80cad60 msgr2=0x7feaf80cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafecdb640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaf80cad60 0x7feaf80cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaf80cad60 0x7feaf80cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 -- 172.21.15.32:0/4253845460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80c9b20 con 0x7feae40047d0 2023-10-16T08:17:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafecdb640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaf80cad60 0x7feaf80cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafdcd9640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feaf80ca8e0 0x7feaf80cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feafe4da640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feaf80b0130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feaf4008460 tx=0x7feaf4005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4008670 con 0x7feae40047d0 2023-10-16T08:17:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4004140 con 0x7feae40047d0 2023-10-16T08:17:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf80c9e00 con 0x7feae40047d0 2023-10-16T08:17:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40044d0 con 0x7feae40047d0 2023-10-16T08:17:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf80b0670 con 0x7feae40047d0 2023-10-16T08:17:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.569+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf80b0860 con 0x7feae40047d0 2023-10-16T08:17:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.573+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feaf401e3b0 con 0x7feae40047d0 2023-10-16T08:17:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.573+0000 7feaeeffd640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feadc0664a0 0x7feadc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.573+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feaf4084390 con 0x7feae40047d0 2023-10-16T08:17:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.573+0000 7feafdcd9640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feadc0664a0 0x7feadc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.573+0000 7feafdcd9640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feadc0664a0 0x7feadc068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feae80b0000 tx=0x7feae80aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.577+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feaf80b0860 con 0x7feae40047d0 2023-10-16T08:17:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feaf80b0860 con 0x7feae40047d0 2023-10-16T08:17:21.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaeeffd640 1 -- 172.21.15.32:0/4253845460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7feaf80b0860 con 0x7feae40047d0 2023-10-16T08:17:21.739 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feadc0664a0 msgr2=0x7feadc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feadc0664a0 0x7feadc068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feae80b0000 tx=0x7feae80aa070 comp rx=0 tx=0).stop 2023-10-16T08:17:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 msgr2=0x7feaf80b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feaf80b0130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feaf4008460 tx=0x7feaf4005b80 comp rx=0 tx=0).stop 2023-10-16T08:17:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 shutdown_connections 2023-10-16T08:17:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feaf80cad60 0x7feaf80cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7feadc0664a0 0x7feadc068960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feaf80ca8e0 0x7feaf80cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 --2- 172.21.15.32:0/4253845460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feae40047d0 0x7feaf80b0130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 >> 172.21.15.32:0/4253845460 conn(0x7feaf80d2800 msgr2=0x7feaf806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 shutdown_connections 2023-10-16T08:17:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.733+0000 7feaff4dc640 1 -- 172.21.15.32:0/4253845460 wait complete. 2023-10-16T08:17:21.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 Processor -- start 2023-10-16T08:17:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 -- start start 2023-10-16T08:17:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f655c15cba0 0x7f655c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f655c0b84e0 0x7f655c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c0d0040 0x7f655c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655c0d1460 con 0x7f655c15cba0 2023-10-16T08:17:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655c0d04b0 con 0x7f655c0b84e0 2023-10-16T08:17:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6560e6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0d1670 con 0x7f655c0d0040 2023-10-16T08:17:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f655c15cba0 0x7f655c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f655c15cba0 0x7f655c0d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43306/0 (socket says 172.21.15.32:43306) 2023-10-16T08:17:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 -- 172.21.15.32:0/1645667215 learned_addr learned my addr 172.21.15.32:0/1645667215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f6559d9b640 1 --2- 172.21.15.32:0/1645667215 >> v2:172.21.15.186:3300/0 conn(0x7f655c0b84e0 0x7f655c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655ad9d640 1 --2- 172.21.15.32:0/1645667215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c0d0040 0x7f655c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 -- 172.21.15.32:0/1645667215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c0d0040 msgr2=0x7f655c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 --2- 172.21.15.32:0/1645667215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c0d0040 0x7f655c0d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 -- 172.21.15.32:0/1645667215 >> v2:172.21.15.186:3300/0 conn(0x7f655c0b84e0 msgr2=0x7f655c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 --2- 172.21.15.32:0/1645667215 >> v2:172.21.15.186:3300/0 conn(0x7f655c0b84e0 0x7f655c0cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.901+0000 7f655a59c640 1 -- 172.21.15.32:0/1645667215 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0d1950 con 0x7f655c15cba0 2023-10-16T08:17:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f655a59c640 1 --2- 172.21.15.32:0/1645667215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c15cba0 0x7f655c0d0f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f655c0cf2f0 tx=0x7f654c01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b808414f6beb3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f655959a640 1 -- 172.21.15.32:0/1645667215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0ab600 con 0x7f655c15cba0 2023-10-16T08:17:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f655959a640 1 -- 172.21.15.32:0/1645667215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c0a7050 con 0x7f655c15cba0 2023-10-16T08:17:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f655959a640 1 -- 172.21.15.32:0/1645667215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0ab980 con 0x7f655c15cba0 2023-10-16T08:17:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 -- 172.21.15.32:0/1645667215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c15cba0 msgr2=0x7f655c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1645667215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c15cba0 0x7f655c0d0f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f655c0cf2f0 tx=0x7f654c01c240 comp rx=0 tx=0).stop 2023-10-16T08:17:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 -- 172.21.15.32:0/1645667215 shutdown_connections 2023-10-16T08:17:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1645667215 >> v2:172.21.15.186:3300/0 conn(0x7f655c0b84e0 0x7f655c0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1645667215 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c0d0040 0x7f655c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1645667215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c15cba0 0x7f655c0d0f20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 -- 172.21.15.32:0/1645667215 >> 172.21.15.32:0/1645667215 conn(0x7f655c0719c0 msgr2=0x7f655c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 -- 172.21.15.32:0/1645667215 shutdown_connections 2023-10-16T08:17:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.905+0000 7f6560e6b640 1 -- 172.21.15.32:0/1645667215 wait complete. 2023-10-16T08:17:21.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 Processor -- start 2023-10-16T08:17:21.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- start start 2023-10-16T08:17:21.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 0x7f655c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f655c0d0040 0x7f655c150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c15cba0 0x7f655c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c155eb0 con 0x7f655c0b84e0 2023-10-16T08:17:21.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c156340 con 0x7f655c0d0040 2023-10-16T08:17:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0c8620 con 0x7f655c15cba0 2023-10-16T08:17:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 0x7f655c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 0x7f655c154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43320/0 (socket says 172.21.15.32:43320) 2023-10-16T08:17:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 -- 172.21.15.32:0/1513261663 learned_addr learned my addr 172.21.15.32:0/1513261663 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6559d9b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f655c0d0040 0x7f655c150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655ad9d640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c15cba0 0x7f655c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 -- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c15cba0 msgr2=0x7f655c0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c15cba0 0x7f655c0c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 -- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f655c0d0040 msgr2=0x7f655c150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f655c0d0040 0x7f655c150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:21.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 -- 172.21.15.32:0/1513261663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c0094d0 con 0x7f655c0b84e0 2023-10-16T08:17:21.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f655a59c640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 0x7f655c154bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f654c0ae040 tx=0x7f654c018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0b4640 con 0x7f655c0b84e0 2023-10-16T08:17:21.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c0a4e90 con 0x7f655c0b84e0 2023-10-16T08:17:21.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c1552a0 con 0x7f655c0b84e0 2023-10-16T08:17:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0b4a00 con 0x7f655c0b84e0 2023-10-16T08:17:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c155860 con 0x7f655c0b84e0 2023-10-16T08:17:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.909+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c04e170 con 0x7f655c0b84e0 2023-10-16T08:17:21.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.913+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f654c0b2070 con 0x7f655c0b84e0 2023-10-16T08:17:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.913+0000 7f654affd640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f653c0664d0 0x7f653c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:21.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.913+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f654c11dfe0 con 0x7f655c0b84e0 2023-10-16T08:17:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.913+0000 7f6559d9b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f653c0664d0 0x7f653c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.913+0000 7f6559d9b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f653c0664d0 0x7f653c068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f655c1f8200 tx=0x7f6550008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:21.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:21.917+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f654c0eb0d0 con 0x7f655c0b84e0 2023-10-16T08:17:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.073+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f655c06a290 con 0x7f655c0b84e0 2023-10-16T08:17:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.073+0000 7f654affd640 1 -- 172.21.15.32:0/1513261663 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f654c0b0070 con 0x7f655c0b84e0 2023-10-16T08:17:22.081 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f653c0664d0 msgr2=0x7f653c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f653c0664d0 0x7f653c068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f655c1f8200 tx=0x7f6550008040 comp rx=0 tx=0).stop 2023-10-16T08:17:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 msgr2=0x7f655c154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 0x7f655c154bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f654c0ae040 tx=0x7f654c018fe0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 shutdown_connections 2023-10-16T08:17:22.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f653c0664d0 0x7f653c068990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f655c15cba0 0x7f655c0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f655c0d0040 0x7f655c150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 --2- 172.21.15.32:0/1513261663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f655c0b84e0 0x7f655c154bc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 >> 172.21.15.32:0/1513261663 conn(0x7f655c0719c0 msgr2=0x7f655c15a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 shutdown_connections 2023-10-16T08:17:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.077+0000 7f6560e6b640 1 -- 172.21.15.32:0/1513261663 wait complete. 2023-10-16T08:17:22.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.241+0000 7f81fdb2a640 1 Processor -- start 2023-10-16T08:17:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.241+0000 7f81fdb2a640 1 -- start start 2023-10-16T08:17:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81fdb2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81f806df40 0x7f81f80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81fdb2a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f81f8070830 0x7f81f80cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81fdb2a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f8070c60 con 0x7f81f81590d0 2023-10-16T08:17:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81fdb2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f80c9600 con 0x7f81f806df40 2023-10-16T08:17:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81fdb2a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f80c98e0 con 0x7f81f8070830 2023-10-16T08:17:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81f806df40 0x7f81f80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81f806df40 0x7f81f80702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43332/0 (socket says 172.21.15.32:43332) 2023-10-16T08:17:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 -- 172.21.15.32:0/2543124662 learned_addr learned my addr 172.21.15.32:0/2543124662 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f77fe640 1 --1- 172.21.15.32:0/2543124662 >> v1:172.21.15.32:6789/0 conn(0x7f81f81590d0 0x7f81f80c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45510/0 (socket says 172.21.15.32:45510) 2023-10-16T08:17:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f7fff640 1 --2- 172.21.15.32:0/2543124662 >> v2:172.21.15.186:3300/0 conn(0x7f81f8070830 0x7f81f80cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 -- 172.21.15.32:0/2543124662 >> v2:172.21.15.186:3300/0 conn(0x7f81f8070830 msgr2=0x7f81f80cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 --2- 172.21.15.32:0/2543124662 >> v2:172.21.15.186:3300/0 conn(0x7f81f8070830 0x7f81f80cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 -- 172.21.15.32:0/2543124662 >> v1:172.21.15.32:6789/0 conn(0x7f81f81590d0 legacy=0x7f81f80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 -- 172.21.15.32:0/2543124662 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f80c9bc0 con 0x7f81f806df40 2023-10-16T08:17:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f67fc640 1 -- 172.21.15.32:0/2543124662 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309394039 0 0) 0x7f81f8070c60 con 0x7f81f81590d0 2023-10-16T08:17:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f6ffd640 1 --2- 172.21.15.32:0/2543124662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f806df40 0x7f81f80702f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81f80adc40 tx=0x7f81ec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e732fd4ff04219 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.245+0000 7f81f67fc640 1 -- 172.21.15.32:0/2543124662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0ab650 con 0x7f81f806df40 2023-10-16T08:17:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f67fc640 1 -- 172.21.15.32:0/2543124662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec0ab7b0 con 0x7f81f806df40 2023-10-16T08:17:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f67fc640 1 -- 172.21.15.32:0/2543124662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0abb40 con 0x7f81f806df40 2023-10-16T08:17:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- 172.21.15.32:0/2543124662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f806df40 msgr2=0x7f81f80702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/2543124662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f806df40 0x7f81f80702f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81f80adc40 tx=0x7f81ec01c4d0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- 172.21.15.32:0/2543124662 shutdown_connections 2023-10-16T08:17:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/2543124662 >> v2:172.21.15.186:3300/0 conn(0x7f81f8070830 0x7f81f80cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/2543124662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f806df40 0x7f81f80702f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- 172.21.15.32:0/2543124662 >> 172.21.15.32:0/2543124662 conn(0x7f81f80b0da0 msgr2=0x7f81f80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- 172.21.15.32:0/2543124662 shutdown_connections 2023-10-16T08:17:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- 172.21.15.32:0/2543124662 wait complete. 2023-10-16T08:17:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 Processor -- start 2023-10-16T08:17:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- start start 2023-10-16T08:17:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81f806df40 0x7f81f8152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81f8070830 0x7f81f80ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 0x7f81f80ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80c5fa0 con 0x7f81f81590d0 2023-10-16T08:17:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81536a0 con 0x7f81f8070830 2023-10-16T08:17:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81fdb2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f8153f40 con 0x7f81f806df40 2023-10-16T08:17:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 0x7f81f80ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f77fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81f806df40 0x7f81f8152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 0x7f81f80ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43342/0 (socket says 172.21.15.32:43342) 2023-10-16T08:17:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f7fff640 1 -- 172.21.15.32:0/598214098 learned_addr learned my addr 172.21.15.32:0/598214098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.249+0000 7f81f6ffd640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81f8070830 0x7f81f80ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81f7fff640 1 -- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81f806df40 msgr2=0x7f81f8152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81f7fff640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81f806df40 0x7f81f8152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81f7fff640 1 -- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81f8070830 msgr2=0x7f81f80ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81f7fff640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81f8070830 0x7f81f80ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81f7fff640 1 -- 172.21.15.32:0/598214098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0094d0 con 0x7f81f81590d0 2023-10-16T08:17:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81f7fff640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 0x7f81f80ace00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f81f0007e70 tx=0x7f81f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0014620 con 0x7f81f81590d0 2023-10-16T08:17:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f00147a0 con 0x7f81f81590d0 2023-10-16T08:17:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0014b50 con 0x7f81f81590d0 2023-10-16T08:17:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f80ca820 con 0x7f81f81590d0 2023-10-16T08:17:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f80c5cc0 con 0x7f81f81590d0 2023-10-16T08:17:22.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f804e170 con 0x7f81f81590d0 2023-10-16T08:17:22.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81f0014cb0 con 0x7f81f81590d0 2023-10-16T08:17:22.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81dffff640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.253+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f81f0084b40 con 0x7f81f81590d0 2023-10-16T08:17:22.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.257+0000 7f81f77fe640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.257+0000 7f81f77fe640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f81e8007fd0 tx=0x7f81e8007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.257+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f81f0051cb0 con 0x7f81f81590d0 2023-10-16T08:17:22.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.413+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f81f8154080 con 0x7f81f81590d0 2023-10-16T08:17:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81dffff640 1 -- 172.21.15.32:0/598214098 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f81f00552d0 con 0x7f81f81590d0 2023-10-16T08:17:22.422 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 msgr2=0x7f81e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f81e8007fd0 tx=0x7f81e8007d60 comp rx=0 tx=0).stop 2023-10-16T08:17:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 msgr2=0x7f81f80ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 0x7f81f80ace00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f81f0007e70 tx=0x7f81f000e4b0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 shutdown_connections 2023-10-16T08:17:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f81590d0 0x7f81f80ace00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81f8070830 0x7f81f80ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 --2- 172.21.15.32:0/598214098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81f806df40 0x7f81f8152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 >> 172.21.15.32:0/598214098 conn(0x7f81f80b0da0 msgr2=0x7f81f80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 shutdown_connections 2023-10-16T08:17:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.417+0000 7f81fdb2a640 1 -- 172.21.15.32:0/598214098 wait complete. 2023-10-16T08:17:22.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 Processor -- start 2023-10-16T08:17:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 -- start start 2023-10-16T08:17:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f81b80d1490 0x7f81b806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81b815cd60 0x7f81b80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80d3510 con 0x7f81b815c980 2023-10-16T08:17:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b80cd2b0 con 0x7f81b815cd60 2023-10-16T08:17:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bde74640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b80d2f50 con 0x7f81b80d1490 2023-10-16T08:17:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81b815cd60 0x7f81b80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81b815cd60 0x7f81b80b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43352/0 (socket says 172.21.15.32:43352) 2023-10-16T08:17:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 -- 172.21.15.32:0/825549125 learned_addr learned my addr 172.21.15.32:0/825549125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81b7fff640 1 --2- 172.21.15.32:0/825549125 >> v2:172.21.15.186:3300/0 conn(0x7f81b80d1490 0x7f81b806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 -- 172.21.15.32:0/825549125 >> v2:172.21.15.186:3300/0 conn(0x7f81b80d1490 msgr2=0x7f81b806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 --2- 172.21.15.32:0/825549125 >> v2:172.21.15.186:3300/0 conn(0x7f81b80d1490 0x7f81b806af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 -- 172.21.15.32:0/825549125 >> v1:172.21.15.186:6789/0 conn(0x7f81b815c980 legacy=0x7f81b80d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.581+0000 7f81bd673640 1 -- 172.21.15.32:0/825549125 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80d3230 con 0x7f81b815cd60 2023-10-16T08:17:22.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bd673640 1 --2- 172.21.15.32:0/825549125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815cd60 0x7f81b80b1b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f81b80d1900 tx=0x7f81b001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1a350746b78268 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81b77fe640 1 -- 172.21.15.32:0/825549125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00a5710 con 0x7f81b815cd60 2023-10-16T08:17:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81b77fe640 1 -- 172.21.15.32:0/825549125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b00a5870 con 0x7f81b815cd60 2023-10-16T08:17:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81b77fe640 1 -- 172.21.15.32:0/825549125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00a5c00 con 0x7f81b815cd60 2023-10-16T08:17:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 -- 172.21.15.32:0/825549125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815cd60 msgr2=0x7f81b80b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 --2- 172.21.15.32:0/825549125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815cd60 0x7f81b80b1b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f81b80d1900 tx=0x7f81b001d3a0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 -- 172.21.15.32:0/825549125 shutdown_connections 2023-10-16T08:17:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 --2- 172.21.15.32:0/825549125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815cd60 0x7f81b80b1b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 --2- 172.21.15.32:0/825549125 >> v2:172.21.15.186:3300/0 conn(0x7f81b80d1490 0x7f81b806af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 -- 172.21.15.32:0/825549125 >> 172.21.15.32:0/825549125 conn(0x7f81b809ba80 msgr2=0x7f81b80d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 -- 172.21.15.32:0/825549125 shutdown_connections 2023-10-16T08:17:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.585+0000 7f81bde74640 1 -- 172.21.15.32:0/825549125 wait complete. 2023-10-16T08:17:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 Processor -- start 2023-10-16T08:17:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 -- start start 2023-10-16T08:17:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81b80d1490 0x7f81b8155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 0x7f81b80afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81b815cd60 0x7f81b8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8155a40 con 0x7f81b815c980 2023-10-16T08:17:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8154a60 con 0x7f81b815cd60 2023-10-16T08:17:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8158100 con 0x7f81b80d1490 2023-10-16T08:17:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 0x7f81b80afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 0x7f81b80afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43354/0 (socket says 172.21.15.32:43354) 2023-10-16T08:17:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 -- 172.21.15.32:0/2610357098 learned_addr learned my addr 172.21.15.32:0/2610357098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 -- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81b80d1490 msgr2=0x7f81b8155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81b80d1490 0x7f81b8155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 -- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81b815cd60 msgr2=0x7f81b8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81b815cd60 0x7f81b8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 -- 172.21.15.32:0/2610357098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009b30 con 0x7f81b815c980 2023-10-16T08:17:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b7fff640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 0x7f81b80afc10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d800 tx=0x7f81a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400b6c0 con 0x7f81b815c980 2023-10-16T08:17:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400b820 con 0x7f81b815c980 2023-10-16T08:17:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b81583e0 con 0x7f81b815c980 2023-10-16T08:17:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400bbb0 con 0x7f81b815c980 2023-10-16T08:17:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.589+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b8154fe0 con 0x7f81b815c980 2023-10-16T08:17:22.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.593+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b80b3c50 con 0x7f81b815c980 2023-10-16T08:17:22.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.593+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81a400bd10 con 0x7f81b815c980 2023-10-16T08:17:22.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.593+0000 7f81b57fa640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f819c0664a0 0x7f819c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:22.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.593+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f81a4084760 con 0x7f81b815c980 2023-10-16T08:17:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.593+0000 7f81bce72640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f819c0664a0 0x7f819c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.593+0000 7f81bce72640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f819c0664a0 0x7f819c068960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000fc0 tx=0x7f81ac0079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.597+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f81a4051820 con 0x7f81b815c980 2023-10-16T08:17:22.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.765+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f81b804e9c0 con 0x7f81b815c980 2023-10-16T08:17:22.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.765+0000 7f81b57fa640 1 -- 172.21.15.32:0/2610357098 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f81a4054e40 con 0x7f81b815c980 2023-10-16T08:17:22.772 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:22.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f819c0664a0 msgr2=0x7f819c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f819c0664a0 0x7f819c068960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000fc0 tx=0x7f81ac0079b0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 msgr2=0x7f81b80afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:22.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 0x7f81b80afc10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d800 tx=0x7f81a400dcc0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 shutdown_connections 2023-10-16T08:17:22.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81b815cd60 0x7f81b8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b815c980 0x7f81b80afc10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f819c0664a0 0x7f819c068960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 --2- 172.21.15.32:0/2610357098 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f81b80d1490 0x7f81b8155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 >> 172.21.15.32:0/2610357098 conn(0x7f81b809ba80 msgr2=0x7f81b80b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 shutdown_connections 2023-10-16T08:17:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:22.769+0000 7f81bde74640 1 -- 172.21.15.32:0/2610357098 wait complete. 2023-10-16T08:17:22.794 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:17:22.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:22.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:22.816 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:22.816 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:17:22.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:22.817 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:17:22.870 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:17:22.871 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:22.881 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:17:22.882 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:22.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:23.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.069+0000 7fecdb2d6640 1 Processor -- start 2023-10-16T08:17:23.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.069+0000 7fecdb2d6640 1 -- start start 2023-10-16T08:17:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecdb2d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecd40cf8e0 0x7fecd40d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecdb2d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40d30b0 con 0x7fecd40ce5e0 2023-10-16T08:17:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecdb2d6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40d3390 con 0x7fecd4160320 2023-10-16T08:17:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecdb2d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40d3670 con 0x7fecd40cf8e0 2023-10-16T08:17:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd8871640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecd40cf8e0 0x7fecd40d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd9873640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fecd40ce5e0 0x7fecd40d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45694/0 (socket says 172.21.15.32:45694) 2023-10-16T08:17:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd9873640 1 -- 172.21.15.32:0/3500994713 learned_addr learned my addr 172.21.15.32:0/3500994713 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd8871640 1 -- 172.21.15.32:0/3500994713 >> v1:172.21.15.186:6789/0 conn(0x7fecd4160320 legacy=0x7fecd40cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd8871640 1 -- 172.21.15.32:0/3500994713 >> v1:172.21.15.32:6789/0 conn(0x7fecd40ce5e0 legacy=0x7fecd40d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd8871640 1 -- 172.21.15.32:0/3500994713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40ce0c0 con 0x7fecd40cf8e0 2023-10-16T08:17:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7feccbfff640 1 -- 172.21.15.32:0/3500994713 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367213486 0 0) 0x7fecd40d30b0 con 0x7fecd40ce5e0 2023-10-16T08:17:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.073+0000 7fecd8871640 1 --2- 172.21.15.32:0/3500994713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd40cf8e0 0x7fecd40d04d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecc40097c0 tx=0x7fecc401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbd736096ce6b8f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7feccbfff640 1 -- 172.21.15.32:0/3500994713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc40ab600 con 0x7fecd40cf8e0 2023-10-16T08:17:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7feccbfff640 1 -- 172.21.15.32:0/3500994713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc40a7050 con 0x7fecd40cf8e0 2023-10-16T08:17:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7feccbfff640 1 -- 172.21.15.32:0/3500994713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc40ab980 con 0x7fecd40cf8e0 2023-10-16T08:17:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- 172.21.15.32:0/3500994713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd40cf8e0 msgr2=0x7fecd40d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/3500994713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd40cf8e0 0x7fecd40d04d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecc40097c0 tx=0x7fecc401c3f0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- 172.21.15.32:0/3500994713 shutdown_connections 2023-10-16T08:17:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/3500994713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd40cf8e0 0x7fecd40d04d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- 172.21.15.32:0/3500994713 >> 172.21.15.32:0/3500994713 conn(0x7fecd409bc50 msgr2=0x7fecd40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- 172.21.15.32:0/3500994713 shutdown_connections 2023-10-16T08:17:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- 172.21.15.32:0/3500994713 wait complete. 2023-10-16T08:17:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 Processor -- start 2023-10-16T08:17:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- start start 2023-10-16T08:17:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecd40ce5e0 0x7fecd406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecd40cf8e0 0x7fecd4148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 0x7fecd40af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd415ab70 con 0x7fecd4160320 2023-10-16T08:17:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4156f20 con 0x7fecd40ce5e0 2023-10-16T08:17:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecdb2d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4157200 con 0x7fecd40cf8e0 2023-10-16T08:17:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecd9873640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 0x7fecd40af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 0x7fecd40af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40424/0 (socket says 172.21.15.32:40424) 2023-10-16T08:17:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.077+0000 7fecd9072640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecd40ce5e0 0x7fecd406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd8871640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecd40cf8e0 0x7fecd4148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 -- 172.21.15.32:0/2106421450 learned_addr learned my addr 172.21.15.32:0/2106421450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 -- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecd40cf8e0 msgr2=0x7fecd4148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecd40cf8e0 0x7fecd4148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 -- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecd40ce5e0 msgr2=0x7fecd406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecd40ce5e0 0x7fecd406cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 -- 172.21.15.32:0/2106421450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4009420 con 0x7fecd4160320 2023-10-16T08:17:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecd9873640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 0x7fecd40af820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feccc001570 tx=0x7feccc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc00a640 con 0x7fecd4160320 2023-10-16T08:17:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc00a7c0 con 0x7fecd4160320 2023-10-16T08:17:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc00ab70 con 0x7fecd4160320 2023-10-16T08:17:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd41574e0 con 0x7fecd4160320 2023-10-16T08:17:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd40afd60 con 0x7fecd4160320 2023-10-16T08:17:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd4074010 con 0x7fecd4160320 2023-10-16T08:17:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.081+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feccc00acd0 con 0x7fecd4160320 2023-10-16T08:17:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.085+0000 7fecc9ffb640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fecb80664a0 0x7fecb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.085+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feccc083800 con 0x7fecd4160320 2023-10-16T08:17:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.085+0000 7fecd9072640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fecb80664a0 0x7fecb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.085+0000 7fecd9072640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fecb80664a0 0x7fecb8068960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecd0000fc0 tx=0x7fecd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.085+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feccc0508d0 con 0x7fecd4160320 2023-10-16T08:17:23.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.245+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecd415acb0 con 0x7fecd4160320 2023-10-16T08:17:23.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecc9ffb640 1 -- 172.21.15.32:0/2106421450 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7feccc018090 con 0x7fecd4160320 2023-10-16T08:17:23.255 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:23.255 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fecb80664a0 msgr2=0x7fecb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fecb80664a0 0x7fecb8068960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecd0000fc0 tx=0x7fecd0009040 comp rx=0 tx=0).stop 2023-10-16T08:17:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 msgr2=0x7fecd40af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 0x7fecd40af820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feccc001570 tx=0x7feccc00cca0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 shutdown_connections 2023-10-16T08:17:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecd4160320 0x7fecd40af820 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fecb80664a0 0x7fecb8068960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecd40cf8e0 0x7fecd4148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 --2- 172.21.15.32:0/2106421450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecd40ce5e0 0x7fecd406cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 >> 172.21.15.32:0/2106421450 conn(0x7fecd409bc50 msgr2=0x7fecd406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 shutdown_connections 2023-10-16T08:17:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.249+0000 7fecdb2d6640 1 -- 172.21.15.32:0/2106421450 wait complete. 2023-10-16T08:17:23.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:23.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 Processor -- start 2023-10-16T08:17:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 -- start start 2023-10-16T08:17:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaa815cc60 0x7faaa80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7faaa80ceda0 0x7faaa80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa80cc330 con 0x7faaa8161260 2023-10-16T08:17:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa80cb720 con 0x7faaa815cc60 2023-10-16T08:17:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaae1d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa80cba00 con 0x7faaa80ceda0 2023-10-16T08:17:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaa815cc60 0x7faaa80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaa815cc60 0x7faaa80cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40438/0 (socket says 172.21.15.32:40438) 2023-10-16T08:17:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 -- 172.21.15.32:0/3192201234 learned_addr learned my addr 172.21.15.32:0/3192201234 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa7fff640 1 --2- 172.21.15.32:0/3192201234 >> v2:172.21.15.186:3300/0 conn(0x7faaa80ceda0 0x7faaa80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 -- 172.21.15.32:0/3192201234 >> v2:172.21.15.186:3300/0 conn(0x7faaa80ceda0 msgr2=0x7faaa80cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 --2- 172.21.15.32:0/3192201234 >> v2:172.21.15.186:3300/0 conn(0x7faaa80ceda0 0x7faaa80cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 -- 172.21.15.32:0/3192201234 >> v1:172.21.15.186:6789/0 conn(0x7faaa8161260 legacy=0x7faaa80ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.417+0000 7faaa77fe640 1 -- 172.21.15.32:0/3192201234 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80cbce0 con 0x7faaa815cc60 2023-10-16T08:17:23.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaa77fe640 1 --2- 172.21.15.32:0/3192201234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa80cb040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faaa8072030 tx=0x7faa9801cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4515a3840dacbc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaa67fc640 1 -- 172.21.15.32:0/3192201234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa980aa610 con 0x7faaa815cc60 2023-10-16T08:17:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaa67fc640 1 -- 172.21.15.32:0/3192201234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa980a6040 con 0x7faaa815cc60 2023-10-16T08:17:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaa67fc640 1 -- 172.21.15.32:0/3192201234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa980aa990 con 0x7faaa815cc60 2023-10-16T08:17:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 -- 172.21.15.32:0/3192201234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 msgr2=0x7faaa80cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3192201234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa80cb040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faaa8072030 tx=0x7faa9801cef0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 -- 172.21.15.32:0/3192201234 shutdown_connections 2023-10-16T08:17:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3192201234 >> v2:172.21.15.186:3300/0 conn(0x7faaa80ceda0 0x7faaa80cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3192201234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa80cb040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 -- 172.21.15.32:0/3192201234 >> 172.21.15.32:0/3192201234 conn(0x7faaa806e570 msgr2=0x7faaa806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 -- 172.21.15.32:0/3192201234 shutdown_connections 2023-10-16T08:17:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.421+0000 7faaae1d9640 1 -- 172.21.15.32:0/3192201234 wait complete. 2023-10-16T08:17:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 Processor -- start 2023-10-16T08:17:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 -- start start 2023-10-16T08:17:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faaa80ceda0 0x7faaa81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa8161260 0x7faaa815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80ca830 con 0x7faaa815cc60 2023-10-16T08:17:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80c9c20 con 0x7faaa8161260 2023-10-16T08:17:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80c9f00 con 0x7faaa80ceda0 2023-10-16T08:17:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40450/0 (socket says 172.21.15.32:40450) 2023-10-16T08:17:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 -- 172.21.15.32:0/3567263888 learned_addr learned my addr 172.21.15.32:0/3567263888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa77fe640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faaa80ceda0 0x7faaa81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa7fff640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa8161260 0x7faaa815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 -- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faaa80ceda0 msgr2=0x7faaa81594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faaa80ceda0 0x7faaa81594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 -- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa8161260 msgr2=0x7faaa815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa8161260 0x7faaa815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 -- 172.21.15.32:0/3567263888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980094d0 con 0x7faaa815cc60 2023-10-16T08:17:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaa6ffd640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa815af60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00b410 tx=0x7faa9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c004020 con 0x7faaa815cc60 2023-10-16T08:17:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c004180 con 0x7faaa815cc60 2023-10-16T08:17:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c011630 con 0x7faaa815cc60 2023-10-16T08:17:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa80ca1e0 con 0x7faaa815cc60 2023-10-16T08:17:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.425+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa8159a30 con 0x7faaa815cc60 2023-10-16T08:17:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.429+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa804e170 con 0x7faaa815cc60 2023-10-16T08:17:23.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.429+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faa9c011790 con 0x7faaa815cc60 2023-10-16T08:17:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.429+0000 7faa8ffff640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faa940664a0 0x7faa94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.429+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faa9c083f20 con 0x7faaa815cc60 2023-10-16T08:17:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.429+0000 7faaa77fe640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faa940664a0 0x7faa94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.429+0000 7faaa77fe640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faa940664a0 0x7faa94068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa980ae000 tx=0x7faa980b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.433+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faa9c050ff0 con 0x7faaa815cc60 2023-10-16T08:17:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.589+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faaa80cdc20 con 0x7faaa815cc60 2023-10-16T08:17:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.589+0000 7faa8ffff640 1 -- 172.21.15.32:0/3567263888 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa9c0150a0 con 0x7faaa815cc60 2023-10-16T08:17:23.597 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faa940664a0 msgr2=0x7faa94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faa940664a0 0x7faa94068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa980ae000 tx=0x7faa980b5040 comp rx=0 tx=0).stop 2023-10-16T08:17:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 msgr2=0x7faaa815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa815af60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00b410 tx=0x7faa9c00b8d0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 shutdown_connections 2023-10-16T08:17:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faaa8161260 0x7faaa815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa815cc60 0x7faaa815af60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7faa940664a0 0x7faa94068960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 --2- 172.21.15.32:0/3567263888 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7faaa80ceda0 0x7faaa81594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 >> 172.21.15.32:0/3567263888 conn(0x7faaa806e570 msgr2=0x7faaa806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 shutdown_connections 2023-10-16T08:17:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.593+0000 7faaae1d9640 1 -- 172.21.15.32:0/3567263888 wait complete. 2023-10-16T08:17:23.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 Processor -- start 2023-10-16T08:17:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 -- start start 2023-10-16T08:17:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c15cba0 0x7fcf4c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcf4c0b84e0 0x7fcf4c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf4c0d0040 0x7fcf4c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c0d1460 con 0x7fcf4c0d0040 2023-10-16T08:17:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c0d04b0 con 0x7fcf4c0b84e0 2023-10-16T08:17:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf522d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0d1670 con 0x7fcf4c15cba0 2023-10-16T08:17:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf4c0d0040 0x7fcf4c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf4c0d0040 0x7fcf4c0d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40462/0 (socket says 172.21.15.32:40462) 2023-10-16T08:17:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 -- 172.21.15.32:0/2226166115 learned_addr learned my addr 172.21.15.32:0/2226166115 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf4b7fe640 1 --2- 172.21.15.32:0/2226166115 >> v2:172.21.15.186:3300/0 conn(0x7fcf4c0b84e0 0x7fcf4c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf4bfff640 1 --2- 172.21.15.32:0/2226166115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c15cba0 0x7fcf4c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 -- 172.21.15.32:0/2226166115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c15cba0 msgr2=0x7fcf4c0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:23.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 --2- 172.21.15.32:0/2226166115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c15cba0 0x7fcf4c0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 -- 172.21.15.32:0/2226166115 >> v2:172.21.15.186:3300/0 conn(0x7fcf4c0b84e0 msgr2=0x7fcf4c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 --2- 172.21.15.32:0/2226166115 >> v2:172.21.15.186:3300/0 conn(0x7fcf4c0b84e0 0x7fcf4c0cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.757+0000 7fcf50873640 1 -- 172.21.15.32:0/2226166115 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0d1950 con 0x7fcf4c0d0040 2023-10-16T08:17:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf50873640 1 --2- 172.21.15.32:0/2226166115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c0d2ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0d3bb0 tx=0x7fcf4401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ded7a5b8c43877b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf4affd640 1 -- 172.21.15.32:0/2226166115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440a5590 con 0x7fcf4c0d0040 2023-10-16T08:17:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf4affd640 1 -- 172.21.15.32:0/2226166115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf440a56f0 con 0x7fcf4c0d0040 2023-10-16T08:17:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf4affd640 1 -- 172.21.15.32:0/2226166115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440a5a80 con 0x7fcf4c0d0040 2023-10-16T08:17:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 -- 172.21.15.32:0/2226166115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c0d0040 msgr2=0x7fcf4c0d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 --2- 172.21.15.32:0/2226166115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c0d2ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0d3bb0 tx=0x7fcf4401c710 comp rx=0 tx=0).stop 2023-10-16T08:17:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 -- 172.21.15.32:0/2226166115 shutdown_connections 2023-10-16T08:17:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 --2- 172.21.15.32:0/2226166115 >> v2:172.21.15.186:3300/0 conn(0x7fcf4c0b84e0 0x7fcf4c0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 --2- 172.21.15.32:0/2226166115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c0d2ae0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 --2- 172.21.15.32:0/2226166115 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c15cba0 0x7fcf4c0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 -- 172.21.15.32:0/2226166115 >> 172.21.15.32:0/2226166115 conn(0x7fcf4c0719c0 msgr2=0x7fcf4c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 -- 172.21.15.32:0/2226166115 shutdown_connections 2023-10-16T08:17:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.761+0000 7fcf522d6640 1 -- 172.21.15.32:0/2226166115 wait complete. 2023-10-16T08:17:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 Processor -- start 2023-10-16T08:17:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 -- start start 2023-10-16T08:17:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c0b84e0 0x7fcf4c14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 0x7fcf4c153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c158e00 con 0x7fcf4c15cba0 2023-10-16T08:17:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c14e0f0 con 0x7fcf4c0d0040 2023-10-16T08:17:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1581f0 con 0x7fcf4c0b84e0 2023-10-16T08:17:23.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 0x7fcf4c153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 0x7fcf4c153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40466/0 (socket says 172.21.15.32:40466) 2023-10-16T08:17:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 -- 172.21.15.32:0/1828032483 learned_addr learned my addr 172.21.15.32:0/1828032483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf4bfff640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c0b84e0 0x7fcf4c14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf4b7fe640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 -- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c0b84e0 msgr2=0x7fcf4c14dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c0b84e0 0x7fcf4c14dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 -- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf4c0d0040 msgr2=0x7fcf4c153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 -- 172.21.15.32:0/1828032483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf44009b30 con 0x7fcf4c15cba0 2023-10-16T08:17:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf50873640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 0x7fcf4c153870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcf4401cfb0 tx=0x7fcf440194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440b8670 con 0x7fcf4c15cba0 2023-10-16T08:17:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf440b87f0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440b8ba0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c1584d0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.765+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c158a90 con 0x7fcf4c15cba0 2023-10-16T08:17:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.769+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c158fb0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.769+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf440b8d00 con 0x7fcf4c15cba0 2023-10-16T08:17:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.769+0000 7fcf48ff9640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf380664a0 0x7fcf38068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.769+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcf4411f600 con 0x7fcf4c15cba0 2023-10-16T08:17:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.769+0000 7fcf4bfff640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf380664a0 0x7fcf38068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.769+0000 7fcf4bfff640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf380664a0 0x7fcf38068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00b930 tx=0x7fcf3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:23.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.773+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf4c158fb0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.929+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf4c158fb0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.929+0000 7fcf48ff9640 1 -- 172.21.15.32:0/1828032483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf4c158fb0 con 0x7fcf4c15cba0 2023-10-16T08:17:23.937 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf380664a0 msgr2=0x7fcf38068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf380664a0 0x7fcf38068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00b930 tx=0x7fcf3c008040 comp rx=0 tx=0).stop 2023-10-16T08:17:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 msgr2=0x7fcf4c153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 0x7fcf4c153870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcf4401cfb0 tx=0x7fcf440194b0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 shutdown_connections 2023-10-16T08:17:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf4c15cba0 0x7fcf4c153870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf4c0d0040 0x7fcf4c153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf380664a0 0x7fcf38068960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 --2- 172.21.15.32:0/1828032483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf4c0b84e0 0x7fcf4c14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 >> 172.21.15.32:0/1828032483 conn(0x7fcf4c0719c0 msgr2=0x7fcf4c0b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 shutdown_connections 2023-10-16T08:17:23.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:23.933+0000 7fcf522d6640 1 -- 172.21.15.32:0/1828032483 wait complete. 2023-10-16T08:17:23.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.097+0000 7f25ec8d6640 1 Processor -- start 2023-10-16T08:17:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.097+0000 7f25ec8d6640 1 -- start start 2023-10-16T08:17:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25ec8d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25e80ab870 0x7f25e8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25ec8d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e806a800 con 0x7f25e815cbc0 2023-10-16T08:17:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25ec8d6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e80abd00 con 0x7f25e806a3d0 2023-10-16T08:17:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25ec8d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e8161390 con 0x7f25e80ab870 2023-10-16T08:17:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25e80ab870 0x7f25e8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25e659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25e815cbc0 0x7f25e8160860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45706/0 (socket says 172.21.15.32:45706) 2023-10-16T08:17:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25e659c640 1 -- 172.21.15.32:0/4093023150 learned_addr learned my addr 172.21.15.32:0/4093023150 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25dffff640 1 -- 172.21.15.32:0/4093023150 >> v1:172.21.15.186:6789/0 conn(0x7f25e806a3d0 legacy=0x7f25e80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25dffff640 1 -- 172.21.15.32:0/4093023150 >> v1:172.21.15.32:6789/0 conn(0x7f25e815cbc0 legacy=0x7f25e8160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25dffff640 1 -- 172.21.15.32:0/4093023150 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80ccda0 con 0x7f25e80ab870 2023-10-16T08:17:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.101+0000 7f25e5d9b640 1 -- 172.21.15.32:0/4093023150 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590311361 0 0) 0x7f25e806a800 con 0x7f25e815cbc0 2023-10-16T08:17:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25dffff640 1 --2- 172.21.15.32:0/4093023150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e8069e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f25d00097c0 tx=0x7f25d001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4b96de0610993e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25e5d9b640 1 -- 172.21.15.32:0/4093023150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00ad680 con 0x7f25e80ab870 2023-10-16T08:17:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25e5d9b640 1 -- 172.21.15.32:0/4093023150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d00ad7e0 con 0x7f25e80ab870 2023-10-16T08:17:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25e5d9b640 1 -- 172.21.15.32:0/4093023150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00adb70 con 0x7f25e80ab870 2023-10-16T08:17:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4093023150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 msgr2=0x7f25e8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4093023150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e8069e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f25d00097c0 tx=0x7f25d001c380 comp rx=0 tx=0).stop 2023-10-16T08:17:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4093023150 shutdown_connections 2023-10-16T08:17:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4093023150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e8069e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4093023150 >> 172.21.15.32:0/4093023150 conn(0x7f25e80b0da0 msgr2=0x7f25e80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4093023150 shutdown_connections 2023-10-16T08:17:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4093023150 wait complete. 2023-10-16T08:17:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 Processor -- start 2023-10-16T08:17:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.105+0000 7f25ec8d6640 1 -- start start 2023-10-16T08:17:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e806a3d0 0x7f25e8154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e80cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25e815cbc0 0x7f25e80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815d040 con 0x7f25e80ab870 2023-10-16T08:17:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815d2e0 con 0x7f25e806a3d0 2023-10-16T08:17:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815d5c0 con 0x7f25e815cbc0 2023-10-16T08:17:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e80cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e80cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40476/0 (socket says 172.21.15.32:40476) 2023-10-16T08:17:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 -- 172.21.15.32:0/4214731373 learned_addr learned my addr 172.21.15.32:0/4214731373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25e659c640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e806a3d0 0x7f25e8154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25e6d9d640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25e815cbc0 0x7f25e80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 -- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25e815cbc0 msgr2=0x7f25e80cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25e815cbc0 0x7f25e80cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 -- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e806a3d0 msgr2=0x7f25e8154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e806a3d0 0x7f25e8154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 -- 172.21.15.32:0/4214731373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0009420 con 0x7f25e80ab870 2023-10-16T08:17:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25e659c640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e806a3d0 0x7f25e8154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25dffff640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e80cb150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f25d0018f90 tx=0x7f25d001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00b8420 con 0x7f25e80ab870 2023-10-16T08:17:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d00b8580 con 0x7f25e80ab870 2023-10-16T08:17:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00b88f0 con 0x7f25e80ab870 2023-10-16T08:17:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e815d8a0 con 0x7f25e80ab870 2023-10-16T08:17:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e8154aa0 con 0x7f25e80ab870 2023-10-16T08:17:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e804e170 con 0x7f25e80ab870 2023-10-16T08:17:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.109+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f25d00b4040 con 0x7f25e80ab870 2023-10-16T08:17:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.113+0000 7f25deffd640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f25c80664a0 0x7f25c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.113+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f25d011e990 con 0x7f25e80ab870 2023-10-16T08:17:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.113+0000 7f25e659c640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f25c80664a0 0x7f25c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.113+0000 7f25e659c640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f25c80664a0 0x7f25c8068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000f70 tx=0x7f25d8007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.117+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f25d00eb9b0 con 0x7f25e80ab870 2023-10-16T08:17:24.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f25e815dd50 con 0x7f25e80ab870 2023-10-16T08:17:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25deffd640 1 -- 172.21.15.32:0/4214731373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f25d00eefd0 con 0x7f25e80ab870 2023-10-16T08:17:24.279 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f25c80664a0 msgr2=0x7f25c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f25c80664a0 0x7f25c8068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000f70 tx=0x7f25d8007cb0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 msgr2=0x7f25e80cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e80cb150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f25d0018f90 tx=0x7f25d001abc0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 shutdown_connections 2023-10-16T08:17:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f25c80664a0 0x7f25c8068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f25e815cbc0 0x7f25e80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e80ab870 0x7f25e80cb150 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 --2- 172.21.15.32:0/4214731373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e806a3d0 0x7f25e8154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 >> 172.21.15.32:0/4214731373 conn(0x7f25e80b0da0 msgr2=0x7f25e806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 shutdown_connections 2023-10-16T08:17:24.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.273+0000 7f25ec8d6640 1 -- 172.21.15.32:0/4214731373 wait complete. 2023-10-16T08:17:24.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 Processor -- start 2023-10-16T08:17:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 -- start start 2023-10-16T08:17:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd3d8159900 0x7fd3d806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3d80b3f00 0x7fd3d8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d80b4390 con 0x7fd3d806a3d0 2023-10-16T08:17:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d80cd740 con 0x7fd3d80b3f00 2023-10-16T08:17:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3defd9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d806a840 con 0x7fd3d8159900 2023-10-16T08:17:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3d80b3f00 0x7fd3d8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3d80b3f00 0x7fd3d8069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40484/0 (socket says 172.21.15.32:40484) 2023-10-16T08:17:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 -- 172.21.15.32:0/2220638994 learned_addr learned my addr 172.21.15.32:0/2220638994 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3dcd75640 1 --2- 172.21.15.32:0/2220638994 >> v2:172.21.15.186:3300/0 conn(0x7fd3d8159900 0x7fd3d806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 -- 172.21.15.32:0/2220638994 >> v2:172.21.15.186:3300/0 conn(0x7fd3d8159900 msgr2=0x7fd3d806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 --2- 172.21.15.32:0/2220638994 >> v2:172.21.15.186:3300/0 conn(0x7fd3d8159900 0x7fd3d806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 -- 172.21.15.32:0/2220638994 >> v1:172.21.15.186:6789/0 conn(0x7fd3d806a3d0 legacy=0x7fd3d80cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.441+0000 7fd3d7fff640 1 -- 172.21.15.32:0/2220638994 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d80b4600 con 0x7fd3d80b3f00 2023-10-16T08:17:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3d7fff640 1 --2- 172.21.15.32:0/2220638994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d8069e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3d806ab70 tx=0x7fd3c401e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5a308c6cec5c16 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3d77fe640 1 -- 172.21.15.32:0/2220638994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a86b0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3d77fe640 1 -- 172.21.15.32:0/2220638994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c40a8810 con 0x7fd3d80b3f00 2023-10-16T08:17:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3d77fe640 1 -- 172.21.15.32:0/2220638994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a8ba0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 -- 172.21.15.32:0/2220638994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 msgr2=0x7fd3d8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 --2- 172.21.15.32:0/2220638994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d8069e90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3d806ab70 tx=0x7fd3c401e6e0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 -- 172.21.15.32:0/2220638994 shutdown_connections 2023-10-16T08:17:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 --2- 172.21.15.32:0/2220638994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d8069e90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 --2- 172.21.15.32:0/2220638994 >> v2:172.21.15.186:3300/0 conn(0x7fd3d8159900 0x7fd3d806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 -- 172.21.15.32:0/2220638994 >> 172.21.15.32:0/2220638994 conn(0x7fd3d80b0da0 msgr2=0x7fd3d80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 -- 172.21.15.32:0/2220638994 shutdown_connections 2023-10-16T08:17:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 -- 172.21.15.32:0/2220638994 wait complete. 2023-10-16T08:17:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.445+0000 7fd3defd9640 1 Processor -- start 2023-10-16T08:17:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- start start 2023-10-16T08:17:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3d806a3d0 0x7fd3d8153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3d8159900 0x7fd3d814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d814bff0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d81539b0 con 0x7fd3d806a3d0 2023-10-16T08:17:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d815b290 con 0x7fd3d8159900 2023-10-16T08:17:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d81543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40490/0 (socket says 172.21.15.32:40490) 2023-10-16T08:17:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 -- 172.21.15.32:0/1744980198 learned_addr learned my addr 172.21.15.32:0/1744980198 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3dcd75640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3d806a3d0 0x7fd3d8153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3dd576640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3d8159900 0x7fd3d814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 -- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3d8159900 msgr2=0x7fd3d814bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3d8159900 0x7fd3d814bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 -- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3d806a3d0 msgr2=0x7fd3d8153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3d806a3d0 0x7fd3d8153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 -- 172.21.15.32:0/1744980198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c40094d0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d7fff640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d81543b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4024cd0 tx=0x7fd3c401cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a8e50 con 0x7fd3d80b3f00 2023-10-16T08:17:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c40bb470 con 0x7fd3d80b3f00 2023-10-16T08:17:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d8153c00 con 0x7fd3d80b3f00 2023-10-16T08:17:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40bb820 con 0x7fd3d80b3f00 2023-10-16T08:17:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d815b4e0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.449+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d815b6d0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.453+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd3c40bb980 con 0x7fd3d80b3f00 2023-10-16T08:17:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.453+0000 7fd3d57fa640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3bc0664a0 0x7fd3bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.453+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd3c40ac070 con 0x7fd3d80b3f00 2023-10-16T08:17:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.453+0000 7fd3dcd75640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3bc0664a0 0x7fd3bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.453+0000 7fd3dcd75640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3bc0664a0 0x7fd3bc068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80adb70 tx=0x7fd3cc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.457+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd3c40ee490 con 0x7fd3d80b3f00 2023-10-16T08:17:24.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.613+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd3d804e170 con 0x7fd3d80b3f00 2023-10-16T08:17:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.613+0000 7fd3d57fa640 1 -- 172.21.15.32:0/1744980198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd3c40f1ab0 con 0x7fd3d80b3f00 2023-10-16T08:17:24.622 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:24.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3bc0664a0 msgr2=0x7fd3bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3bc0664a0 0x7fd3bc068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80adb70 tx=0x7fd3cc009190 comp rx=0 tx=0).stop 2023-10-16T08:17:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 msgr2=0x7fd3d81543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d81543b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4024cd0 tx=0x7fd3c401cf20 comp rx=0 tx=0).stop 2023-10-16T08:17:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 shutdown_connections 2023-10-16T08:17:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3bc0664a0 0x7fd3bc068960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3d8159900 0x7fd3d814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d80b3f00 0x7fd3d81543b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 --2- 172.21.15.32:0/1744980198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3d806a3d0 0x7fd3d8153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 >> 172.21.15.32:0/1744980198 conn(0x7fd3d80b0da0 msgr2=0x7fd3d80cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 shutdown_connections 2023-10-16T08:17:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.617+0000 7fd3defd9640 1 -- 172.21.15.32:0/1744980198 wait complete. 2023-10-16T08:17:24.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:24.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.781+0000 7f6f2add0640 1 Processor -- start 2023-10-16T08:17:24.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.781+0000 7f6f2add0640 1 -- start start 2023-10-16T08:17:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2add0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 0x7f6f2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2add0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2415d4e0 con 0x7f6f240ce310 2023-10-16T08:17:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2add0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2415d7c0 con 0x7f6f240b2ad0 2023-10-16T08:17:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2add0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f24160580 con 0x7f6f2415cf90 2023-10-16T08:17:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2936d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f240ce310 0x7f6f241609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45716/0 (socket says 172.21.15.32:45716) 2023-10-16T08:17:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2936d640 1 -- 172.21.15.32:0/4130779520 learned_addr learned my addr 172.21.15.32:0/4130779520 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750599906 0 0) 0x7f6f2415d4e0 con 0x7f6f240ce310 2023-10-16T08:17:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f23fff640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 0x7f6f2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f14003530 con 0x7f6f240ce310 2023-10-16T08:17:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3414198965 0 0) 0x7f6f14003530 con 0x7f6f240ce310 2023-10-16T08:17:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 msgr2=0x7f6f2406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 0x7f6f2406c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 >> v1:172.21.15.186:6789/0 conn(0x7f6f240b2ad0 legacy=0x7f6f2406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2415daa0 con 0x7f6f240ce310 2023-10-16T08:17:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f6f1c001a00 con 0x7f6f240ce310 2023-10-16T08:17:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 >> v1:172.21.15.32:6789/0 conn(0x7f6f240ce310 legacy=0x7f6f241609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140039b0 0x7f6f14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f14004360 0x7f6f14004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 0x7f6f14005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f24160580 con 0x7f6f140039b0 2023-10-16T08:17:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1400b9c0 con 0x7f6f14004360 2023-10-16T08:17:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1400bb80 con 0x7f6f2415cf90 2023-10-16T08:17:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140039b0 0x7f6f14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f23fff640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f14004360 0x7f6f14004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f2936d640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 0x7f6f14005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f28b6c640 1 -- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 msgr2=0x7f6f14005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f2415cf90 0x7f6f14005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f28b6c640 1 -- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f14004360 msgr2=0x7f6f14004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f14004360 0x7f6f14004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f28b6c640 1 -- 172.21.15.32:0/4130779520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f14003530 con 0x7f6f140039b0 2023-10-16T08:17:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.785+0000 7f6f23fff640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f14004360 0x7f6f14004830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140039b0 0x7f6f14003d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6f18001910 tx=0x7f6f1801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fe4b5714131682 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180ad660 con 0x7f6f140039b0 2023-10-16T08:17:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f180ad7e0 con 0x7f6f140039b0 2023-10-16T08:17:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f237fe640 1 -- 172.21.15.32:0/4130779520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180adb90 con 0x7f6f140039b0 2023-10-16T08:17:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 -- 172.21.15.32:0/4130779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140039b0 msgr2=0x7f6f14003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 --2- 172.21.15.32:0/4130779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140039b0 0x7f6f14003d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6f18001910 tx=0x7f6f1801c530 comp rx=0 tx=0).stop 2023-10-16T08:17:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2936d640 1 -- 172.21.15.32:0/4130779520 reap_dead start 2023-10-16T08:17:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 -- 172.21.15.32:0/4130779520 shutdown_connections 2023-10-16T08:17:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 -- 172.21.15.32:0/4130779520 >> 172.21.15.32:0/4130779520 conn(0x7f6f240cd1c0 msgr2=0x7f6f240cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:24.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 -- 172.21.15.32:0/4130779520 shutdown_connections 2023-10-16T08:17:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 -- 172.21.15.32:0/4130779520 wait complete. 2023-10-16T08:17:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.789+0000 7f6f2add0640 1 Processor -- start 2023-10-16T08:17:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- start start 2023-10-16T08:17:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f140039b0 0x7f6f240b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 0x7f6f240b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f240b1230 0x7f6f240b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240b1b30 con 0x7f6f2415c220 2023-10-16T08:17:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2415b740 con 0x7f6f140039b0 2023-10-16T08:17:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2415ba20 con 0x7f6f240b1230 2023-10-16T08:17:24.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 0x7f6f240b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 0x7f6f240b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40502/0 (socket says 172.21.15.32:40502) 2023-10-16T08:17:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 -- 172.21.15.32:0/3661224727 learned_addr learned my addr 172.21.15.32:0/3661224727 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f140039b0 0x7f6f240b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2936d640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f240b1230 0x7f6f240b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 -- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f240b1230 msgr2=0x7f6f240b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f240b1230 0x7f6f240b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 -- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f140039b0 msgr2=0x7f6f240b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f140039b0 0x7f6f240b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 -- 172.21.15.32:0/3661224727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240cecb0 con 0x7f6f2415c220 2023-10-16T08:17:24.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f140039b0 0x7f6f240b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:24.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f23fff640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 0x7f6f240b0cf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009fd0 tx=0x7f6f10011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10004220 con 0x7f6f2415c220 2023-10-16T08:17:24.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10004380 con 0x7f6f2415c220 2023-10-16T08:17:24.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000f9e0 con 0x7f6f2415c220 2023-10-16T08:17:24.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2415bd00 con 0x7f6f2415c220 2023-10-16T08:17:24.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f240b16b0 con 0x7f6f2415c220 2023-10-16T08:17:24.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.793+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2415b5a0 con 0x7f6f2415c220 2023-10-16T08:17:24.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.797+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6f1000fb40 con 0x7f6f2415c220 2023-10-16T08:17:24.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.797+0000 7f6f217fa640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f080664d0 0x7f6f08068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:24.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.797+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6f10084780 con 0x7f6f2415c220 2023-10-16T08:17:24.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.797+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f080664d0 0x7f6f08068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:24.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.797+0000 7f6f28b6c640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f080664d0 0x7f6f08068990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6f180af000 tx=0x7f6f18001330 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:24.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.801+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6f2415b5a0 con 0x7f6f2415c220 2023-10-16T08:17:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.969+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f2415b5a0 con 0x7f6f2415c220 2023-10-16T08:17:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.969+0000 7f6f217fa640 1 -- 172.21.15.32:0/3661224727 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f2415b5a0 con 0x7f6f2415c220 2023-10-16T08:17:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f080664d0 msgr2=0x7f6f08068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f080664d0 0x7f6f08068990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6f180af000 tx=0x7f6f18001330 comp rx=0 tx=0).stop 2023-10-16T08:17:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 msgr2=0x7f6f240b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 0x7f6f240b0cf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009fd0 tx=0x7f6f10011b30 comp rx=0 tx=0).stop 2023-10-16T08:17:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 shutdown_connections 2023-10-16T08:17:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6f080664d0 0x7f6f08068990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6f240b1230 0x7f6f240b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f2415c220 0x7f6f240b0cf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 --2- 172.21.15.32:0/3661224727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f140039b0 0x7f6f240b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:24.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 >> 172.21.15.32:0/3661224727 conn(0x7f6f240cd1c0 msgr2=0x7f6f24162ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:24.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 shutdown_connections 2023-10-16T08:17:24.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:24.973+0000 7f6f2add0640 1 -- 172.21.15.32:0/3661224727 wait complete. 2023-10-16T08:17:24.998 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:17:24.998 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:25.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862fa79640 1 Processor -- start 2023-10-16T08:17:25.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862fa79640 1 -- start start 2023-10-16T08:17:25.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862fa79640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862fa79640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280723d0 con 0x7f8628159ba0 2023-10-16T08:17:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862fa79640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280717c0 con 0x7f86281597c0 2023-10-16T08:17:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862fa79640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8628071aa0 con 0x7f86280c8cb0 2023-10-16T08:17:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862d014640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8628159ba0 0x7f86280cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45726/0 (socket says 172.21.15.32:45726) 2023-10-16T08:17:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862d014640 1 -- 172.21.15.32:0/2858052794 learned_addr learned my addr 172.21.15.32:0/2858052794 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862e016640 1 --2- 172.21.15.32:0/2858052794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574474253 0 0) 0x7f86280723d0 con 0x7f8628159ba0 2023-10-16T08:17:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8614005740 con 0x7f8628159ba0 2023-10-16T08:17:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069843087 0 0) 0x7f86280717c0 con 0x7f86281597c0 2023-10-16T08:17:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86280723d0 con 0x7f86281597c0 2023-10-16T08:17:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653245099 0 0) 0x7f8614005740 con 0x7f8628159ba0 2023-10-16T08:17:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862e016640 1 -- 172.21.15.32:0/2858052794 >> v1:172.21.15.186:6789/0 conn(0x7f86281597c0 legacy=0x7f86280ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862e016640 1 -- 172.21.15.32:0/2858052794 >> v1:172.21.15.32:6789/0 conn(0x7f8628159ba0 legacy=0x7f86280cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.145+0000 7f862e016640 1 -- 172.21.15.32:0/2858052794 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628071d80 con 0x7f86280c8cb0 2023-10-16T08:17:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862e016640 1 --2- 172.21.15.32:0/2858052794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280cb520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8620009dc0 tx=0x7f862001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c53b2a7249dca0a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618001680 con 0x7f86280c8cb0 2023-10-16T08:17:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86200a5570 con 0x7f86280c8cb0 2023-10-16T08:17:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862c813640 1 -- 172.21.15.32:0/2858052794 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200a5900 con 0x7f86280c8cb0 2023-10-16T08:17:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 -- 172.21.15.32:0/2858052794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 msgr2=0x7f86280cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 --2- 172.21.15.32:0/2858052794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280cb520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8620009dc0 tx=0x7f862001c890 comp rx=0 tx=0).stop 2023-10-16T08:17:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 -- 172.21.15.32:0/2858052794 shutdown_connections 2023-10-16T08:17:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 --2- 172.21.15.32:0/2858052794 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280cb520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 -- 172.21.15.32:0/2858052794 >> 172.21.15.32:0/2858052794 conn(0x7f862806c190 msgr2=0x7f862806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 -- 172.21.15.32:0/2858052794 shutdown_connections 2023-10-16T08:17:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.149+0000 7f862fa79640 1 -- 172.21.15.32:0/2858052794 wait complete. 2023-10-16T08:17:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 Processor -- start 2023-10-16T08:17:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 -- start start 2023-10-16T08:17:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86281597c0 0x7f862814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 0x7f86280ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c6650 con 0x7f8628159ba0 2023-10-16T08:17:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280ad400 con 0x7f86281597c0 2023-10-16T08:17:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280acb60 con 0x7f86280c8cb0 2023-10-16T08:17:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 0x7f86280ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 0x7f86280ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40504/0 (socket says 172.21.15.32:40504) 2023-10-16T08:17:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 -- 172.21.15.32:0/2101893114 learned_addr learned my addr 172.21.15.32:0/2101893114 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862d014640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86281597c0 0x7f862814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862d815640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 -- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 msgr2=0x7f86280b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 -- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86281597c0 msgr2=0x7f862814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86281597c0 0x7f862814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 -- 172.21.15.32:0/2101893114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620009a70 con 0x7f8628159ba0 2023-10-16T08:17:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862d014640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86281597c0 0x7f862814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862d815640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862e016640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 0x7f86280ac230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f86200b0040 tx=0x7f8620019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200b9650 con 0x7f8628159ba0 2023-10-16T08:17:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86200b97b0 con 0x7f8628159ba0 2023-10-16T08:17:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86280ace40 con 0x7f8628159ba0 2023-10-16T08:17:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200b9b80 con 0x7f8628159ba0 2023-10-16T08:17:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.153+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86280ac770 con 0x7f8628159ba0 2023-10-16T08:17:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.157+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8628074010 con 0x7f8628159ba0 2023-10-16T08:17:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.157+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86200b9ce0 con 0x7f8628159ba0 2023-10-16T08:17:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.157+0000 7f861e7fc640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f860c0664a0 0x7f860c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.157+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f862011f220 con 0x7f8628159ba0 2023-10-16T08:17:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.157+0000 7f862d815640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f860c0664a0 0x7f860c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.157+0000 7f862d815640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f860c0664a0 0x7f860c068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86240011f0 tx=0x7f8624009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.161+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86200ec240 con 0x7f8628159ba0 2023-10-16T08:17:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.317+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86280c6790 con 0x7f8628159ba0 2023-10-16T08:17:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f861e7fc640 1 -- 172.21.15.32:0/2101893114 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f8620009450 con 0x7f8628159ba0 2023-10-16T08:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:25.327 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f860c0664a0 msgr2=0x7f860c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f860c0664a0 0x7f860c068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86240011f0 tx=0x7f8624009040 comp rx=0 tx=0).stop 2023-10-16T08:17:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 msgr2=0x7f86280ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 0x7f86280ac230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f86200b0040 tx=0x7f8620019630 comp rx=0 tx=0).stop 2023-10-16T08:17:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 shutdown_connections 2023-10-16T08:17:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8628159ba0 0x7f86280ac230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86281597c0 0x7f862814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f860c0664a0 0x7f860c068960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 --2- 172.21.15.32:0/2101893114 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86280c8cb0 0x7f86280b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 >> 172.21.15.32:0/2101893114 conn(0x7f862806c190 msgr2=0x7f86281560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 shutdown_connections 2023-10-16T08:17:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.321+0000 7f862fa79640 1 -- 172.21.15.32:0/2101893114 wait complete. 2023-10-16T08:17:25.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:25.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8049217640 1 Processor -- start 2023-10-16T08:17:25.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8049217640 1 -- start start 2023-10-16T08:17:25.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8049217640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f804415ce10 0x7f8044161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8049217640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440cdc30 con 0x7f80440cd3e0 2023-10-16T08:17:25.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8049217640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440caec0 con 0x7f804406e2f0 2023-10-16T08:17:25.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8049217640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440c9750 con 0x7f804415ce10 2023-10-16T08:17:25.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f804259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f80440cd3e0 0x7f80440ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45732/0 (socket says 172.21.15.32:45732) 2023-10-16T08:17:25.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f804259c640 1 -- 172.21.15.32:0/2581032990 learned_addr learned my addr 172.21.15.32:0/2581032990 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:25.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8042d9d640 1 --2- 172.21.15.32:0/2581032990 >> v2:172.21.15.186:3300/0 conn(0x7f804415ce10 0x7f8044161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041484014 0 0) 0x7f80440cdc30 con 0x7f80440cd3e0 2023-10-16T08:17:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c003530 con 0x7f80440cd3e0 2023-10-16T08:17:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4146921270 0 0) 0x7f802c003530 con 0x7f80440cd3e0 2023-10-16T08:17:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 >> v2:172.21.15.186:3300/0 conn(0x7f804415ce10 msgr2=0x7f8044161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 --2- 172.21.15.32:0/2581032990 >> v2:172.21.15.186:3300/0 conn(0x7f804415ce10 0x7f8044161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 >> v1:172.21.15.186:6789/0 conn(0x7f804406e2f0 legacy=0x7f8044069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440c99f0 con 0x7f80440cd3e0 2023-10-16T08:17:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f8038001680 con 0x7f80440cd3e0 2023-10-16T08:17:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 >> v1:172.21.15.32:6789/0 conn(0x7f80440cd3e0 legacy=0x7f80440ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f802c003980 0x7f802c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0047d0 0x7f802c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c005190 0x7f802c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c9750 con 0x7f802c0047d0 2023-10-16T08:17:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00c030 con 0x7f802c003980 2023-10-16T08:17:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00c1f0 con 0x7f802c005190 2023-10-16T08:17:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.489+0000 7f804259c640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0047d0 0x7f802c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8042d9d640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f802c003980 0x7f802c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804359e640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c005190 0x7f802c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804259c640 1 -- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c005190 msgr2=0x7f802c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804259c640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c005190 0x7f802c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804259c640 1 -- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f802c003980 msgr2=0x7f802c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:25.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804259c640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f802c003980 0x7f802c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804259c640 1 -- 172.21.15.32:0/2581032990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c003530 con 0x7f802c0047d0 2023-10-16T08:17:25.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804359e640 1 -- 172.21.15.32:0/2581032990 reap_dead start 2023-10-16T08:17:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f804259c640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0047d0 0x7f802c004c50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f803800c240 tx=0x7f803801f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b140e63775d2d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038005ae0 con 0x7f802c0047d0 2023-10-16T08:17:25.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80380ab070 con 0x7f802c0047d0 2023-10-16T08:17:25.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8041d9b640 1 -- 172.21.15.32:0/2581032990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038007070 con 0x7f802c0047d0 2023-10-16T08:17:25.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8049217640 1 -- 172.21.15.32:0/2581032990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0047d0 msgr2=0x7f802c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8049217640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0047d0 0x7f802c004c50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f803800c240 tx=0x7f803801f370 comp rx=0 tx=0).stop 2023-10-16T08:17:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8049217640 1 -- 172.21.15.32:0/2581032990 shutdown_connections 2023-10-16T08:17:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8049217640 1 --2- 172.21.15.32:0/2581032990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0047d0 0x7f802c004c50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.493+0000 7f8049217640 1 -- 172.21.15.32:0/2581032990 >> 172.21.15.32:0/2581032990 conn(0x7f80440cef20 msgr2=0x7f80440cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 -- 172.21.15.32:0/2581032990 shutdown_connections 2023-10-16T08:17:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 -- 172.21.15.32:0/2581032990 wait complete. 2023-10-16T08:17:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 Processor -- start 2023-10-16T08:17:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 -- start start 2023-10-16T08:17:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c0047d0 0x7f80440c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044157660 0x7f80441564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 0x7f8044158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044157e40 con 0x7f80441f2540 2023-10-16T08:17:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c6190 con 0x7f8044157660 2023-10-16T08:17:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f8049217640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c6470 con 0x7f802c0047d0 2023-10-16T08:17:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 0x7f8044158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 0x7f8044158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40532/0 (socket says 172.21.15.32:40532) 2023-10-16T08:17:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 -- 172.21.15.32:0/3207802268 learned_addr learned my addr 172.21.15.32:0/3207802268 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 -- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c0047d0 msgr2=0x7f80440c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c0047d0 0x7f80440c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 -- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044157660 msgr2=0x7f80441564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044157660 0x7f80441564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 -- 172.21.15.32:0/3207802268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440c6750 con 0x7f80441f2540 2023-10-16T08:17:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.497+0000 7f804359e640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 0x7f8044158710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8044070970 tx=0x7f803c00fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00fec0 con 0x7f80441f2540 2023-10-16T08:17:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c003b90 con 0x7f80441f2540 2023-10-16T08:17:25.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0050d0 con 0x7f80441f2540 2023-10-16T08:17:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80440c6a30 con 0x7f80441f2540 2023-10-16T08:17:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8044157ae0 con 0x7f80441f2540 2023-10-16T08:17:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80440c6c20 con 0x7f80441f2540 2023-10-16T08:17:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f803c005230 con 0x7f80441f2540 2023-10-16T08:17:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f802b7fe640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80300664d0 0x7f8030068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f803c0841c0 con 0x7f80441f2540 2023-10-16T08:17:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.501+0000 7f8042d9d640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80300664d0 0x7f8030068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.505+0000 7f8042d9d640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80300664d0 0x7f8030068990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8034006e90 tx=0x7f8034006e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.505+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f803c0511e0 con 0x7f80441f2540 2023-10-16T08:17:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.661+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f804404e170 con 0x7f80441f2540 2023-10-16T08:17:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.661+0000 7f802b7fe640 1 -- 172.21.15.32:0/3207802268 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f803c054800 con 0x7f80441f2540 2023-10-16T08:17:25.670 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80300664d0 msgr2=0x7f8030068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80300664d0 0x7f8030068990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8034006e90 tx=0x7f8034006e00 comp rx=0 tx=0).stop 2023-10-16T08:17:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 msgr2=0x7f8044158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 0x7f8044158710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8044070970 tx=0x7f803c00fc70 comp rx=0 tx=0).stop 2023-10-16T08:17:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 shutdown_connections 2023-10-16T08:17:25.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80441f2540 0x7f8044158710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044157660 0x7f80441564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80300664d0 0x7f8030068990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 --2- 172.21.15.32:0/3207802268 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f802c0047d0 0x7f80440c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 >> 172.21.15.32:0/3207802268 conn(0x7f80440cef20 msgr2=0x7f804415d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:25.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 shutdown_connections 2023-10-16T08:17:25.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.665+0000 7f8049217640 1 -- 172.21.15.32:0/3207802268 wait complete. 2023-10-16T08:17:25.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:25.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 Processor -- start 2023-10-16T08:17:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 -- start start 2023-10-16T08:17:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f320006a5a0 0x7f320006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 0x7f32000c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32000b4370 con 0x7f3200159900 2023-10-16T08:17:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32000cd740 con 0x7f320006a5a0 2023-10-16T08:17:25.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.829+0000 7f3206ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32000b4600 con 0x7f32000b3f00 2023-10-16T08:17:25.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f3205073640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 0x7f32000c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f31fffff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f320006a5a0 0x7f320006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f3205073640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 0x7f32000c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:34708/0 (socket says 172.21.15.32:34708) 2023-10-16T08:17:25.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f3205073640 1 -- 172.21.15.32:0/2737584303 learned_addr learned my addr 172.21.15.32:0/2737584303 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:25.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f31fffff640 1 -- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 msgr2=0x7f32000c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:25.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f31fffff640 1 --2- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 0x7f32000c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f31fffff640 1 -- 172.21.15.32:0/2737584303 >> v1:172.21.15.186:6789/0 conn(0x7f3200159900 legacy=0x7f3200069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f31fffff640 1 -- 172.21.15.32:0/2737584303 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000c8dd0 con 0x7f320006a5a0 2023-10-16T08:17:25.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f3205073640 1 --2- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 0x7f32000c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:25.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.833+0000 7f31fffff640 1 --2- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f320006e260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0097c0 tx=0x7f31ec01e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=507eaea497fe0237 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f31ff7fe640 1 -- 172.21.15.32:0/2737584303 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0a8630 con 0x7f320006a5a0 2023-10-16T08:17:25.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f31ff7fe640 1 -- 172.21.15.32:0/2737584303 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec0a8790 con 0x7f320006a5a0 2023-10-16T08:17:25.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f31ff7fe640 1 -- 172.21.15.32:0/2737584303 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0a8b20 con 0x7f320006a5a0 2023-10-16T08:17:25.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 msgr2=0x7f320006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 --2- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f320006e260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0097c0 tx=0x7f31ec01e660 comp rx=0 tx=0).stop 2023-10-16T08:17:25.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- 172.21.15.32:0/2737584303 shutdown_connections 2023-10-16T08:17:25.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 --2- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32000b3f00 0x7f32000c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 --2- 172.21.15.32:0/2737584303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f320006e260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- 172.21.15.32:0/2737584303 >> 172.21.15.32:0/2737584303 conn(0x7f32000b0da0 msgr2=0x7f32000ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:25.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- 172.21.15.32:0/2737584303 shutdown_connections 2023-10-16T08:17:25.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- 172.21.15.32:0/2737584303 wait complete. 2023-10-16T08:17:25.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 Processor -- start 2023-10-16T08:17:25.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- start start 2023-10-16T08:17:25.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f3200158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 0x7f3200153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3200159900 0x7f320015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320015c750 con 0x7f32000b3f00 2023-10-16T08:17:25.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320015ca30 con 0x7f320006a5a0 2023-10-16T08:17:25.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3206ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320015cd10 con 0x7f3200159900 2023-10-16T08:17:25.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f31fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 0x7f3200153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f31fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 0x7f3200153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40542/0 (socket says 172.21.15.32:40542) 2023-10-16T08:17:25.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f31fffff640 1 -- 172.21.15.32:0/3356212297 learned_addr learned my addr 172.21.15.32:0/3356212297 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:25.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3204872640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f3200158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.837+0000 7f3205073640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3200159900 0x7f320015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fffff640 1 -- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3200159900 msgr2=0x7f320015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fffff640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3200159900 0x7f320015a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fffff640 1 -- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 msgr2=0x7f3200158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:25.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fffff640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f3200158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:25.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fffff640 1 -- 172.21.15.32:0/3356212297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009b30 con 0x7f32000b3f00 2023-10-16T08:17:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fffff640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 0x7f3200153810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f31ec01cd10 tx=0x7f31ec01b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0b3650 con 0x7f32000b3f00 2023-10-16T08:17:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec0b37b0 con 0x7f32000b3f00 2023-10-16T08:17:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0b3b60 con 0x7f32000b3f00 2023-10-16T08:17:25.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec009420 con 0x7f32000b3f00 2023-10-16T08:17:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32001f4d70 con 0x7f32000b3f00 2023-10-16T08:17:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320004e170 con 0x7f32000b3f00 2023-10-16T08:17:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31ec0b3cc0 con 0x7f32000b3f00 2023-10-16T08:17:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fd7fa640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f31e40664a0 0x7f31e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.841+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f31ec11e960 con 0x7f32000b3f00 2023-10-16T08:17:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.845+0000 7f3204872640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f31e40664a0 0x7f31e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.845+0000 7f3204872640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f31e40664a0 0x7f31e4068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31f4007fd0 tx=0x7f31f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:25.845+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f31ec0eba30 con 0x7f32000b3f00 2023-10-16T08:17:26.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.005+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f320006d330 con 0x7f32000b3f00 2023-10-16T08:17:26.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.005+0000 7f31fd7fa640 1 -- 172.21.15.32:0/3356212297 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31ec0ef050 con 0x7f32000b3f00 2023-10-16T08:17:26.012 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:26.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.005+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f31e40664a0 msgr2=0x7f31e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.005+0000 7f3206ad6640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f31e40664a0 0x7f31e4068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31f4007fd0 tx=0x7f31f4009040 comp rx=0 tx=0).stop 2023-10-16T08:17:26.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.005+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 msgr2=0x7f3200153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.005+0000 7f3206ad6640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 0x7f3200153810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f31ec01cd10 tx=0x7f31ec01b400 comp rx=0 tx=0).stop 2023-10-16T08:17:26.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 shutdown_connections 2023-10-16T08:17:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f31e40664a0 0x7f31e4068960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3200159900 0x7f320015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32000b3f00 0x7f3200153810 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 --2- 172.21.15.32:0/3356212297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320006a5a0 0x7f3200158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 >> 172.21.15.32:0/3356212297 conn(0x7f32000b0da0 msgr2=0x7f3200158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:26.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 shutdown_connections 2023-10-16T08:17:26.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.009+0000 7f3206ad6640 1 -- 172.21.15.32:0/3356212297 wait complete. 2023-10-16T08:17:26.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91e00d6640 1 Processor -- start 2023-10-16T08:17:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91e00d6640 1 -- start start 2023-10-16T08:17:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91e00d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91d80d0210 0x7f91d80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91e00d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80d1670 con 0x7f91d815cba0 2023-10-16T08:17:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91e00d6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80d1950 con 0x7f91d80b84e0 2023-10-16T08:17:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91e00d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80d1460 con 0x7f91d80d0210 2023-10-16T08:17:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dd671640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91d80d0210 0x7f91d80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dde72640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91d815cba0 0x7f91d80cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45748/0 (socket says 172.21.15.32:45748) 2023-10-16T08:17:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dd671640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91d80d0210 0x7f91d80d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40554/0 (socket says 172.21.15.32:40554) 2023-10-16T08:17:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dde72640 1 -- 172.21.15.32:0/4186071135 learned_addr learned my addr 172.21.15.32:0/4186071135 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dd671640 1 -- 172.21.15.32:0/4186071135 >> v1:172.21.15.186:6789/0 conn(0x7f91d80b84e0 legacy=0x7f91d80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dd671640 1 -- 172.21.15.32:0/4186071135 >> v1:172.21.15.32:6789/0 conn(0x7f91d815cba0 legacy=0x7f91d80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dd671640 1 -- 172.21.15.32:0/4186071135 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80d1c30 con 0x7f91d80d0210 2023-10-16T08:17:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.173+0000 7f91dce70640 1 -- 172.21.15.32:0/4186071135 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181866447 0 0) 0x7f91d80d1670 con 0x7f91d815cba0 2023-10-16T08:17:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91dd671640 1 --2- 172.21.15.32:0/4186071135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80d0210 0x7f91d80d0f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91c80097c0 tx=0x7f91c801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842aedafac0bd658 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91dce70640 1 -- 172.21.15.32:0/4186071135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80ac410 con 0x7f91d80d0210 2023-10-16T08:17:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91dce70640 1 -- 172.21.15.32:0/4186071135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c80ac570 con 0x7f91d80d0210 2023-10-16T08:17:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91dce70640 1 -- 172.21.15.32:0/4186071135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80ac900 con 0x7f91d80d0210 2023-10-16T08:17:26.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 -- 172.21.15.32:0/4186071135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80d0210 msgr2=0x7f91d80d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 --2- 172.21.15.32:0/4186071135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80d0210 0x7f91d80d0f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91c80097c0 tx=0x7f91c801c3b0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 -- 172.21.15.32:0/4186071135 shutdown_connections 2023-10-16T08:17:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 --2- 172.21.15.32:0/4186071135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80d0210 0x7f91d80d0f20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 -- 172.21.15.32:0/4186071135 >> 172.21.15.32:0/4186071135 conn(0x7f91d80719c0 msgr2=0x7f91d8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 -- 172.21.15.32:0/4186071135 shutdown_connections 2023-10-16T08:17:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.177+0000 7f91e00d6640 1 -- 172.21.15.32:0/4186071135 wait complete. 2023-10-16T08:17:26.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 Processor -- start 2023-10-16T08:17:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 -- start start 2023-10-16T08:17:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 0x7f91d815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d80d0210 0x7f91d8152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d815cba0 0x7f91d814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d81515a0 con 0x7f91d80b84e0 2023-10-16T08:17:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8151880 con 0x7f91d80d0210 2023-10-16T08:17:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8151b60 con 0x7f91d815cba0 2023-10-16T08:17:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 0x7f91d815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 0x7f91d815e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40564/0 (socket says 172.21.15.32:40564) 2023-10-16T08:17:26.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 -- 172.21.15.32:0/3539398496 learned_addr learned my addr 172.21.15.32:0/3539398496 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91de673640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d815cba0 0x7f91d814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dd671640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d80d0210 0x7f91d8152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 -- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d815cba0 msgr2=0x7f91d814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d815cba0 0x7f91d814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 -- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d80d0210 msgr2=0x7f91d8152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d80d0210 0x7f91d8152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 -- 172.21.15.32:0/3539398496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009420 con 0x7f91d80b84e0 2023-10-16T08:17:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91de673640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d815cba0 0x7f91d814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:26.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91dde72640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 0x7f91d815e5c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c480 tx=0x7f91d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40074e0 con 0x7f91d80b84e0 2023-10-16T08:17:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d4007660 con 0x7f91d80b84e0 2023-10-16T08:17:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004a30 con 0x7f91d80b84e0 2023-10-16T08:17:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8151e40 con 0x7f91d80b84e0 2023-10-16T08:17:26.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.181+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d8152400 con 0x7f91d80b84e0 2023-10-16T08:17:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.185+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d804e170 con 0x7f91d80b84e0 2023-10-16T08:17:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.185+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f91d4004b90 con 0x7f91d80b84e0 2023-10-16T08:17:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.185+0000 7f91ce7fc640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91bc0664a0 0x7f91bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.185+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f91d4083310 con 0x7f91d80b84e0 2023-10-16T08:17:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.185+0000 7f91dd671640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91bc0664a0 0x7f91bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.185+0000 7f91dd671640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91bc0664a0 0x7f91bc068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91c80ae000 tx=0x7f91c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.189+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f91d4053fa0 con 0x7f91d80b84e0 2023-10-16T08:17:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.345+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91d80b7bf0 con 0x7f91d80b84e0 2023-10-16T08:17:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.345+0000 7f91ce7fc640 1 -- 172.21.15.32:0/3539398496 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91d4053b30 con 0x7f91d80b84e0 2023-10-16T08:17:26.353 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91bc0664a0 msgr2=0x7f91bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91bc0664a0 0x7f91bc068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91c80ae000 tx=0x7f91c8008040 comp rx=0 tx=0).stop 2023-10-16T08:17:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 msgr2=0x7f91d815e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 0x7f91d815e5c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c480 tx=0x7f91d400c940 comp rx=0 tx=0).stop 2023-10-16T08:17:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 shutdown_connections 2023-10-16T08:17:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91bc0664a0 0x7f91bc068960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d815cba0 0x7f91d814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d80d0210 0x7f91d8152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 --2- 172.21.15.32:0/3539398496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d80b84e0 0x7f91d815e5c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 >> 172.21.15.32:0/3539398496 conn(0x7f91d80719c0 msgr2=0x7f91d815b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:26.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 shutdown_connections 2023-10-16T08:17:26.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.349+0000 7f91e00d6640 1 -- 172.21.15.32:0/3539398496 wait complete. 2023-10-16T08:17:26.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 Processor -- start 2023-10-16T08:17:26.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 -- start start 2023-10-16T08:17:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d415cba0 0x7ff2d40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff2d40d0210 0x7ff2d40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d40d1670 con 0x7ff2d40b84e0 2023-10-16T08:17:26.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40d1460 con 0x7ff2d40d0210 2023-10-16T08:17:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.513+0000 7ff2d999d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40d1950 con 0x7ff2d415cba0 2023-10-16T08:17:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2cbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff2d40b84e0 0x7ff2d40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45756/0 (socket says 172.21.15.32:45756) 2023-10-16T08:17:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2cbfff640 1 -- 172.21.15.32:0/1587416262 learned_addr learned my addr 172.21.15.32:0/1587416262 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:26.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d2ffd640 1 --2- 172.21.15.32:0/1587416262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d415cba0 0x7ff2d40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d37fe640 1 --2- 172.21.15.32:0/1587416262 >> v2:172.21.15.186:3300/0 conn(0x7ff2d40d0210 0x7ff2d40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d27fc640 1 -- 172.21.15.32:0/1587416262 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928539600 0 0) 0x7ff2d40d1670 con 0x7ff2d40b84e0 2023-10-16T08:17:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d27fc640 1 -- 172.21.15.32:0/1587416262 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c0003530 con 0x7ff2d40b84e0 2023-10-16T08:17:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d2ffd640 1 -- 172.21.15.32:0/1587416262 >> v2:172.21.15.186:3300/0 conn(0x7ff2d40d0210 msgr2=0x7ff2d40b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:26.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d2ffd640 1 --2- 172.21.15.32:0/1587416262 >> v2:172.21.15.186:3300/0 conn(0x7ff2d40d0210 0x7ff2d40b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d2ffd640 1 -- 172.21.15.32:0/1587416262 >> v1:172.21.15.32:6789/0 conn(0x7ff2d40b84e0 legacy=0x7ff2d40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d2ffd640 1 -- 172.21.15.32:0/1587416262 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40d1c30 con 0x7ff2d415cba0 2023-10-16T08:17:26.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d37fe640 1 --2- 172.21.15.32:0/1587416262 >> v2:172.21.15.186:3300/0 conn(0x7ff2d40d0210 0x7ff2d40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:26.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.517+0000 7ff2d2ffd640 1 --2- 172.21.15.32:0/1587416262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d415cba0 0x7ff2d40d0f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097c0 tx=0x7ff2c401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af331af74b9532f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d27fc640 1 -- 172.21.15.32:0/1587416262 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40ab680 con 0x7ff2d415cba0 2023-10-16T08:17:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d27fc640 1 -- 172.21.15.32:0/1587416262 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40ab7e0 con 0x7ff2d415cba0 2023-10-16T08:17:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d27fc640 1 -- 172.21.15.32:0/1587416262 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40abb70 con 0x7ff2d415cba0 2023-10-16T08:17:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- 172.21.15.32:0/1587416262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d415cba0 msgr2=0x7ff2d40d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1587416262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d415cba0 0x7ff2d40d0f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097c0 tx=0x7ff2c401c480 comp rx=0 tx=0).stop 2023-10-16T08:17:26.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- 172.21.15.32:0/1587416262 shutdown_connections 2023-10-16T08:17:26.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1587416262 >> v2:172.21.15.186:3300/0 conn(0x7ff2d40d0210 0x7ff2d40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1587416262 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d415cba0 0x7ff2d40d0f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- 172.21.15.32:0/1587416262 >> 172.21.15.32:0/1587416262 conn(0x7ff2d40719c0 msgr2=0x7ff2d4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:26.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- 172.21.15.32:0/1587416262 shutdown_connections 2023-10-16T08:17:26.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- 172.21.15.32:0/1587416262 wait complete. 2023-10-16T08:17:26.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 Processor -- start 2023-10-16T08:17:26.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- start start 2023-10-16T08:17:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2d40b84e0 0x7ff2d40c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d40d0210 0x7ff2d414db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 0x7ff2d4150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415d280 con 0x7ff2d415cba0 2023-10-16T08:17:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415d4d0 con 0x7ff2d40b84e0 2023-10-16T08:17:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d999d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415d7b0 con 0x7ff2d40d0210 2023-10-16T08:17:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 0x7ff2d4150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 0x7ff2d4150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40572/0 (socket says 172.21.15.32:40572) 2023-10-16T08:17:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d37fe640 1 -- 172.21.15.32:0/1181313701 learned_addr learned my addr 172.21.15.32:0/1181313701 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2d2ffd640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2d40b84e0 0x7ff2d40c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.521+0000 7ff2cbfff640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d40d0210 0x7ff2d414db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d37fe640 1 -- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d40d0210 msgr2=0x7ff2d414db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d37fe640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d40d0210 0x7ff2d414db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d37fe640 1 -- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2d40b84e0 msgr2=0x7ff2d40c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d37fe640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2d40b84e0 0x7ff2d40c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d37fe640 1 -- 172.21.15.32:0/1181313701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc009a70 con 0x7ff2d415cba0 2023-10-16T08:17:26.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d37fe640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 0x7ff2d4150900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00efc0 tx=0x7ff2cc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc009d70 con 0x7ff2d415cba0 2023-10-16T08:17:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc00f040 con 0x7ff2d415cba0 2023-10-16T08:17:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0039a0 con 0x7ff2d415cba0 2023-10-16T08:17:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c4009420 con 0x7ff2d415cba0 2023-10-16T08:17:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d415dd70 con 0x7ff2d415cba0 2023-10-16T08:17:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d415df90 con 0x7ff2d415cba0 2023-10-16T08:17:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff2cc0144a0 con 0x7ff2d415cba0 2023-10-16T08:17:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.525+0000 7ff2d0ff9640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2b40664a0 0x7ff2b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.529+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff2cc084ef0 con 0x7ff2d415cba0 2023-10-16T08:17:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.529+0000 7ff2d2ffd640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2b40664a0 0x7ff2b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.529+0000 7ff2d2ffd640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2b40664a0 0x7ff2b4068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af000 tx=0x7ff2c40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.533+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff2cc051f10 con 0x7ff2d415cba0 2023-10-16T08:17:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.685+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2d404e170 con 0x7ff2d415cba0 2023-10-16T08:17:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.685+0000 7ff2d0ff9640 1 -- 172.21.15.32:0/1181313701 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff2cc0180a0 con 0x7ff2d415cba0 2023-10-16T08:17:26.694 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2b40664a0 msgr2=0x7ff2b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2b40664a0 0x7ff2b4068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af000 tx=0x7ff2c40a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 msgr2=0x7ff2d4150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 0x7ff2d4150900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00efc0 tx=0x7ff2cc009f90 comp rx=0 tx=0).stop 2023-10-16T08:17:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 shutdown_connections 2023-10-16T08:17:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2d415cba0 0x7ff2d4150900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2b40664a0 0x7ff2b4068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2d40d0210 0x7ff2d414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 --2- 172.21.15.32:0/1181313701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2d40b84e0 0x7ff2d40c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 >> 172.21.15.32:0/1181313701 conn(0x7ff2d40719c0 msgr2=0x7ff2d40d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 shutdown_connections 2023-10-16T08:17:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.689+0000 7ff2d999d640 1 -- 172.21.15.32:0/1181313701 wait complete. 2023-10-16T08:17:26.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 Processor -- start 2023-10-16T08:17:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 -- start start 2023-10-16T08:17:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 0x7f41e40d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41e40d3170 0x7f41e40cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e40ce730 con 0x7f41e40cefe0 2023-10-16T08:17:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e40d17a0 con 0x7f41e40d3170 2023-10-16T08:17:26.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41ea474640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40d0dd0 con 0x7f41e415c910 2023-10-16T08:17:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 0x7f41e40d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e8c71640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41e40d3170 0x7f41e40cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 0x7f41e40d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:34746/0 (socket says 172.21.15.32:34746) 2023-10-16T08:17:26.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 -- 172.21.15.32:0/4287796790 learned_addr learned my addr 172.21.15.32:0/4287796790 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 -- 172.21.15.32:0/4287796790 >> v2:172.21.15.186:3300/0 conn(0x7f41e40d3170 msgr2=0x7f41e40cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 --2- 172.21.15.32:0/4287796790 >> v2:172.21.15.186:3300/0 conn(0x7f41e40d3170 0x7f41e40cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 -- 172.21.15.32:0/4287796790 >> v1:172.21.15.186:6789/0 conn(0x7f41e40cefe0 legacy=0x7f41e406b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.853+0000 7f41e9472640 1 -- 172.21.15.32:0/4287796790 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e40d04a0 con 0x7f41e415c910 2023-10-16T08:17:26.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41e9472640 1 --2- 172.21.15.32:0/4287796790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 0x7f41e40d1260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e001c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98146652e6447488 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41dbfff640 1 -- 172.21.15.32:0/4287796790 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00ab660 con 0x7f41e415c910 2023-10-16T08:17:26.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41dbfff640 1 -- 172.21.15.32:0/4287796790 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e00ab7c0 con 0x7f41e415c910 2023-10-16T08:17:26.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41dbfff640 1 -- 172.21.15.32:0/4287796790 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00abb50 con 0x7f41e415c910 2023-10-16T08:17:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 -- 172.21.15.32:0/4287796790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 msgr2=0x7f41e40d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 --2- 172.21.15.32:0/4287796790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 0x7f41e40d1260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e001c460 comp rx=0 tx=0).stop 2023-10-16T08:17:26.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 -- 172.21.15.32:0/4287796790 shutdown_connections 2023-10-16T08:17:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 --2- 172.21.15.32:0/4287796790 >> v2:172.21.15.186:3300/0 conn(0x7f41e40d3170 0x7f41e40cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 --2- 172.21.15.32:0/4287796790 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e415c910 0x7f41e40d1260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 -- 172.21.15.32:0/4287796790 >> 172.21.15.32:0/4287796790 conn(0x7f41e409bb50 msgr2=0x7f41e40d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 -- 172.21.15.32:0/4287796790 shutdown_connections 2023-10-16T08:17:26.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.857+0000 7f41ea474640 1 -- 172.21.15.32:0/4287796790 wait complete. 2023-10-16T08:17:26.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 Processor -- start 2023-10-16T08:17:26.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 -- start start 2023-10-16T08:17:26.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 0x7f41e4152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e40d3170 0x7f41e41551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e415c910 0x7f41e41556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e4150580 con 0x7f41e40cefe0 2023-10-16T08:17:26.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e414fce0 con 0x7f41e415c910 2023-10-16T08:17:26.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e414ffc0 con 0x7f41e40d3170 2023-10-16T08:17:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 0x7f41e4152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 0x7f41e4152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40576/0 (socket says 172.21.15.32:40576) 2023-10-16T08:17:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 -- 172.21.15.32:0/4182257397 learned_addr learned my addr 172.21.15.32:0/4182257397 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 -- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e40d3170 msgr2=0x7f41e41551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e8c71640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e40d3170 0x7f41e41551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9c73640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e415c910 0x7f41e41556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e40d3170 0x7f41e41551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 -- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e415c910 msgr2=0x7f41e41556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e415c910 0x7f41e41556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 -- 172.21.15.32:0/4182257397 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0009480 con 0x7f41e40cefe0 2023-10-16T08:17:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9c73640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e415c910 0x7f41e41556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e8c71640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e40d3170 0x7f41e41551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41e9472640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 0x7f41e4152200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f41e001a090 tx=0x7f41e0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0005db0 con 0x7f41e40cefe0 2023-10-16T08:17:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e00ab5e0 con 0x7f41e40cefe0 2023-10-16T08:17:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00abb50 con 0x7f41e40cefe0 2023-10-16T08:17:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e41502a0 con 0x7f41e40cefe0 2023-10-16T08:17:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.861+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e4151990 con 0x7f41e40cefe0 2023-10-16T08:17:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.865+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e404e9c0 con 0x7f41e40cefe0 2023-10-16T08:17:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.865+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f41e00a9070 con 0x7f41e40cefe0 2023-10-16T08:17:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.865+0000 7f41d9ffb640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41c80664a0 0x7f41c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.865+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f41e011f290 con 0x7f41e40cefe0 2023-10-16T08:17:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.865+0000 7f41e8c71640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41c80664a0 0x7f41c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:26.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.865+0000 7f41e8c71640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41c80664a0 0x7f41c8068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41e4158420 tx=0x7f41d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:26.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:26.869+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f41e00ec360 con 0x7f41e40cefe0 2023-10-16T08:17:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41e41560a0 con 0x7f41e40cefe0 2023-10-16T08:17:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41d9ffb640 1 -- 172.21.15.32:0/4182257397 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f41e00ef980 con 0x7f41e40cefe0 2023-10-16T08:17:27.051 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:27.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41c80664a0 msgr2=0x7f41c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41c80664a0 0x7f41c8068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41e4158420 tx=0x7f41d4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:27.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 msgr2=0x7f41e4152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 0x7f41e4152200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f41e001a090 tx=0x7f41e0019200 comp rx=0 tx=0).stop 2023-10-16T08:17:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 shutdown_connections 2023-10-16T08:17:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41e415c910 0x7f41e41556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41c80664a0 0x7f41c8068960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41e40d3170 0x7f41e41551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 --2- 172.21.15.32:0/4182257397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e40cefe0 0x7f41e4152200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 >> 172.21.15.32:0/4182257397 conn(0x7f41e409bb50 msgr2=0x7f41e40afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:27.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 shutdown_connections 2023-10-16T08:17:27.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.045+0000 7f41ea474640 1 -- 172.21.15.32:0/4182257397 wait complete. 2023-10-16T08:17:27.072 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi186.front.sepia.ceph.com... 2023-10-16T08:17:27.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:27.073 DEBUG:teuthology.orchestra.run.smithi186:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-16T08:17:27.106 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:27.107 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:17:27.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:27.108 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T08:17:27.195 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:17:27.195 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:27.205 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:17:27.205 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:27.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f9068cd6640 1 Processor -- start 2023-10-16T08:17:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f9068cd6640 1 -- start start 2023-10-16T08:17:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f9068cd6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90641597c0 0x7f90640ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f9068cd6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90640723d0 con 0x7f9064159ba0 2023-10-16T08:17:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f9068cd6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90640717c0 con 0x7f90640adc10 2023-10-16T08:17:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f9068cd6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9064071aa0 con 0x7f90641597c0 2023-10-16T08:17:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f905bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9064159ba0 0x7f90640cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45768/0 (socket says 172.21.15.32:45768) 2023-10-16T08:17:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f905bfff640 1 -- 172.21.15.32:0/11614135 learned_addr learned my addr 172.21.15.32:0/11614135 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.437+0000 7f906259c640 1 --2- 172.21.15.32:0/11614135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90641597c0 0x7f90640ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f9061d9b640 1 -- 172.21.15.32:0/11614135 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538011268 0 0) 0x7f90640723d0 con 0x7f9064159ba0 2023-10-16T08:17:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f9061d9b640 1 -- 172.21.15.32:0/11614135 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9050005740 con 0x7f9064159ba0 2023-10-16T08:17:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f906259c640 1 -- 172.21.15.32:0/11614135 >> v1:172.21.15.186:6789/0 conn(0x7f90640adc10 legacy=0x7f906415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f906259c640 1 -- 172.21.15.32:0/11614135 >> v1:172.21.15.32:6789/0 conn(0x7f9064159ba0 legacy=0x7f90640cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f906259c640 1 -- 172.21.15.32:0/11614135 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064071d80 con 0x7f90641597c0 2023-10-16T08:17:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f906259c640 1 --2- 172.21.15.32:0/11614135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90641597c0 0x7f90640ad6d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90540097f0 tx=0x7f905401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931203ff3b4348c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f9061d9b640 1 -- 172.21.15.32:0/11614135 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540a7070 con 0x7f90641597c0 2023-10-16T08:17:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f9061d9b640 1 -- 172.21.15.32:0/11614135 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90540ab430 con 0x7f90641597c0 2023-10-16T08:17:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.441+0000 7f9061d9b640 1 -- 172.21.15.32:0/11614135 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540ab7c0 con 0x7f90641597c0 2023-10-16T08:17:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- 172.21.15.32:0/11614135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90641597c0 msgr2=0x7f90640ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 --2- 172.21.15.32:0/11614135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90641597c0 0x7f90640ad6d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90540097f0 tx=0x7f905401c490 comp rx=0 tx=0).stop 2023-10-16T08:17:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- 172.21.15.32:0/11614135 shutdown_connections 2023-10-16T08:17:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 --2- 172.21.15.32:0/11614135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90641597c0 0x7f90640ad6d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- 172.21.15.32:0/11614135 >> 172.21.15.32:0/11614135 conn(0x7f906406c190 msgr2=0x7f906406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- 172.21.15.32:0/11614135 shutdown_connections 2023-10-16T08:17:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- 172.21.15.32:0/11614135 wait complete. 2023-10-16T08:17:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 Processor -- start 2023-10-16T08:17:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- start start 2023-10-16T08:17:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90640adc10 0x7f90641475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 0x7f906415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9064159ba0 0x7f9064148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c86e0 con 0x7f90641597c0 2023-10-16T08:17:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906415b680 con 0x7f9064159ba0 2023-10-16T08:17:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9068cd6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9064147f00 con 0x7f90640adc10 2023-10-16T08:17:27.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 0x7f906415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 0x7f906415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40588/0 (socket says 172.21.15.32:40588) 2023-10-16T08:17:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 -- 172.21.15.32:0/3141953885 learned_addr learned my addr 172.21.15.32:0/3141953885 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:27.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f906259c640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90640adc10 0x7f90641475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9062d9d640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9064159ba0 0x7f9064148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 -- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90640adc10 msgr2=0x7f90641475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90640adc10 0x7f90641475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 -- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9064159ba0 msgr2=0x7f9064148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9064159ba0 0x7f9064148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f905bfff640 1 -- 172.21.15.32:0/3141953885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9054009420 con 0x7f90641597c0 2023-10-16T08:17:27.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.445+0000 7f9062d9d640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9064159ba0 0x7f9064148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:27.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905bfff640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 0x7f906415b140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c420 tx=0x7f904c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:27.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c005cf0 con 0x7f90641597c0 2023-10-16T08:17:27.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c005e50 con 0x7f90641597c0 2023-10-16T08:17:27.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0072d0 con 0x7f90641597c0 2023-10-16T08:17:27.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90640c7ad0 con 0x7f90641597c0 2023-10-16T08:17:27.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90640c8090 con 0x7f90641597c0 2023-10-16T08:17:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90640c8850 con 0x7f90641597c0 2023-10-16T08:17:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f904c007430 con 0x7f90641597c0 2023-10-16T08:17:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905affd640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90440664a0 0x7f9044068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f904c083870 con 0x7f90641597c0 2023-10-16T08:17:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.449+0000 7f906259c640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90440664a0 0x7f9044068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.453+0000 7f906259c640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90440664a0 0x7f9044068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90540af000 tx=0x7f90540a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:27.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.453+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f904c0508b0 con 0x7f90641597c0 2023-10-16T08:17:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.609+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9064074010 con 0x7f90641597c0 2023-10-16T08:17:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f905affd640 1 -- 172.21.15.32:0/3141953885 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f904c017090 con 0x7f90641597c0 2023-10-16T08:17:27.619 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:27.619 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90440664a0 msgr2=0x7f9044068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90440664a0 0x7f9044068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90540af000 tx=0x7f90540a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 msgr2=0x7f906415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 0x7f906415b140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c420 tx=0x7f904c00c8e0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 shutdown_connections 2023-10-16T08:17:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9064159ba0 0x7f9064148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90641597c0 0x7f906415b140 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90440664a0 0x7f9044068960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 --2- 172.21.15.32:0/3141953885 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90640adc10 0x7f90641475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 >> 172.21.15.32:0/3141953885 conn(0x7f906406c190 msgr2=0x7f906406c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 shutdown_connections 2023-10-16T08:17:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.613+0000 7f9068cd6640 1 -- 172.21.15.32:0/3141953885 wait complete. 2023-10-16T08:17:27.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:27.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:27.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb57135640 1 Processor -- start 2023-10-16T08:17:27.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb57135640 1 -- start start 2023-10-16T08:17:27.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb57135640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb500b84e0 0x7fdb500d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb57135640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb500d1670 con 0x7fdb5015cba0 2023-10-16T08:17:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb57135640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb500d1950 con 0x7fdb500d0210 2023-10-16T08:17:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb57135640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb500d1460 con 0x7fdb500b84e0 2023-10-16T08:17:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb556d2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb500b84e0 0x7fdb500d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb556d2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb500b84e0 0x7fdb500d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40594/0 (socket says 172.21.15.32:40594) 2023-10-16T08:17:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb556d2640 1 -- 172.21.15.32:0/3545708429 learned_addr learned my addr 172.21.15.32:0/3545708429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb556d2640 1 -- 172.21.15.32:0/3545708429 >> v1:172.21.15.186:6789/0 conn(0x7fdb500d0210 legacy=0x7fdb500b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb556d2640 1 -- 172.21.15.32:0/3545708429 >> v1:172.21.15.32:6789/0 conn(0x7fdb5015cba0 legacy=0x7fdb500cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb556d2640 1 -- 172.21.15.32:0/3545708429 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500d1c30 con 0x7fdb500b84e0 2023-10-16T08:17:27.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.781+0000 7fdb4f7fe640 1 -- 172.21.15.32:0/3545708429 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001821341 0 0) 0x7fdb500d1670 con 0x7fdb5015cba0 2023-10-16T08:17:27.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb556d2640 1 --2- 172.21.15.32:0/3545708429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb500d0f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009e50 tx=0x7fdb4801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb2cc64de482bee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:27.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb4f7fe640 1 -- 172.21.15.32:0/3545708429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a5840 con 0x7fdb500b84e0 2023-10-16T08:17:27.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb4f7fe640 1 -- 172.21.15.32:0/3545708429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb480a59a0 con 0x7fdb500b84e0 2023-10-16T08:17:27.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb4f7fe640 1 -- 172.21.15.32:0/3545708429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a5d30 con 0x7fdb500b84e0 2023-10-16T08:17:27.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 -- 172.21.15.32:0/3545708429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 msgr2=0x7fdb500d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 --2- 172.21.15.32:0/3545708429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb500d0f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009e50 tx=0x7fdb4801c980 comp rx=0 tx=0).stop 2023-10-16T08:17:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 -- 172.21.15.32:0/3545708429 shutdown_connections 2023-10-16T08:17:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 --2- 172.21.15.32:0/3545708429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb500d0f20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 -- 172.21.15.32:0/3545708429 >> 172.21.15.32:0/3545708429 conn(0x7fdb500719c0 msgr2=0x7fdb50071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:27.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 -- 172.21.15.32:0/3545708429 shutdown_connections 2023-10-16T08:17:27.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.785+0000 7fdb57135640 1 -- 172.21.15.32:0/3545708429 wait complete. 2023-10-16T08:17:27.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 Processor -- start 2023-10-16T08:17:27.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- start start 2023-10-16T08:17:27.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb501482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb500d0210 0x7fdb5014b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb5015cba0 0x7fdb501520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb50150450 con 0x7fdb500b84e0 2023-10-16T08:17:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5015e430 con 0x7fdb500d0210 2023-10-16T08:17:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5015d8b0 con 0x7fdb5015cba0 2023-10-16T08:17:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb501482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb501482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40608/0 (socket says 172.21.15.32:40608) 2023-10-16T08:17:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 -- 172.21.15.32:0/4041067744 learned_addr learned my addr 172.21.15.32:0/4041067744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb4ffff640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb500d0210 0x7fdb5014b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb556d2640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb5015cba0 0x7fdb501520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 -- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb5015cba0 msgr2=0x7fdb501520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb5015cba0 0x7fdb501520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 -- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb500d0210 msgr2=0x7fdb5014b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb500d0210 0x7fdb5014b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 -- 172.21.15.32:0/4041067744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48009b30 con 0x7fdb500b84e0 2023-10-16T08:17:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb54ed1640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb501482a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb44001ec0 tx=0x7fdb4400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440054e0 con 0x7fdb500b84e0 2023-10-16T08:17:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb44003600 con 0x7fdb500b84e0 2023-10-16T08:17:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440039b0 con 0x7fdb500b84e0 2023-10-16T08:17:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5015db90 con 0x7fdb500b84e0 2023-10-16T08:17:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5015e150 con 0x7fdb500b84e0 2023-10-16T08:17:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.789+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb501505c0 con 0x7fdb500b84e0 2023-10-16T08:17:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.793+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdb44003b10 con 0x7fdb500b84e0 2023-10-16T08:17:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.793+0000 7fdb4d7fa640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb340664a0 0x7fdb34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.793+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdb44083830 con 0x7fdb500b84e0 2023-10-16T08:17:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.793+0000 7fdb4ffff640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb340664a0 0x7fdb34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:27.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.793+0000 7fdb4ffff640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb340664a0 0x7fdb34068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb5014c730 tx=0x7fdb3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:27.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.797+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdb440508e0 con 0x7fdb500b84e0 2023-10-16T08:17:27.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.953+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb5004e170 con 0x7fdb500b84e0 2023-10-16T08:17:27.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb4d7fa640 1 -- 172.21.15.32:0/4041067744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb440170a0 con 0x7fdb500b84e0 2023-10-16T08:17:27.963 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:27.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb340664a0 msgr2=0x7fdb34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb340664a0 0x7fdb34068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb5014c730 tx=0x7fdb3c009040 comp rx=0 tx=0).stop 2023-10-16T08:17:27.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 msgr2=0x7fdb501482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb501482a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb44001ec0 tx=0x7fdb4400c8e0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 shutdown_connections 2023-10-16T08:17:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb340664a0 0x7fdb34068960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb5015cba0 0x7fdb501520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb500d0210 0x7fdb5014b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 --2- 172.21.15.32:0/4041067744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb500b84e0 0x7fdb501482a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 >> 172.21.15.32:0/4041067744 conn(0x7fdb500719c0 msgr2=0x7fdb500b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 shutdown_connections 2023-10-16T08:17:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:27.957+0000 7fdb57135640 1 -- 172.21.15.32:0/4041067744 wait complete. 2023-10-16T08:17:27.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:28.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 Processor -- start 2023-10-16T08:17:28.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 -- start start 2023-10-16T08:17:28.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe2ec0d0210 0x7fe2ec0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2ec0b84e0 0x7fe2ec0b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec0d1670 con 0x7fe2ec15cba0 2023-10-16T08:17:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ec0d1460 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2f133e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ec0d1950 con 0x7fe2ec0d0210 2023-10-16T08:17:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2ec0b84e0 0x7fe2ec0b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2ec0b84e0 0x7fe2ec0b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40620/0 (socket says 172.21.15.32:40620) 2023-10-16T08:17:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eaffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2ec15cba0 0x7fe2ec0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45794/0 (socket says 172.21.15.32:45794) 2023-10-16T08:17:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 -- 172.21.15.32:0/1736747371 learned_addr learned my addr 172.21.15.32:0/1736747371 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2e3fff640 1 --2- 172.21.15.32:0/1736747371 >> v2:172.21.15.186:3300/0 conn(0x7fe2ec0d0210 0x7fe2ec0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 -- 172.21.15.32:0/1736747371 >> v2:172.21.15.186:3300/0 conn(0x7fe2ec0d0210 msgr2=0x7fe2ec0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 --2- 172.21.15.32:0/1736747371 >> v2:172.21.15.186:3300/0 conn(0x7fe2ec0d0210 0x7fe2ec0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 -- 172.21.15.32:0/1736747371 >> v1:172.21.15.32:6789/0 conn(0x7fe2ec15cba0 legacy=0x7fe2ec0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.125+0000 7fe2eb7fe640 1 -- 172.21.15.32:0/1736747371 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0d1c30 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2eb7fe640 1 --2- 172.21.15.32:0/1736747371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec0b5cf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0cf010 tx=0x7fe2e401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55407e6fa558f8c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2ea7fc640 1 -- 172.21.15.32:0/1736747371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40a5800 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2ea7fc640 1 -- 172.21.15.32:0/1736747371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e40a5960 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2ea7fc640 1 -- 172.21.15.32:0/1736747371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40a5cf0 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 -- 172.21.15.32:0/1736747371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 msgr2=0x7fe2ec0b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 --2- 172.21.15.32:0/1736747371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec0b5cf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0cf010 tx=0x7fe2e401c940 comp rx=0 tx=0).stop 2023-10-16T08:17:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 -- 172.21.15.32:0/1736747371 shutdown_connections 2023-10-16T08:17:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 --2- 172.21.15.32:0/1736747371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec0b5cf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 --2- 172.21.15.32:0/1736747371 >> v2:172.21.15.186:3300/0 conn(0x7fe2ec0d0210 0x7fe2ec0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 -- 172.21.15.32:0/1736747371 >> 172.21.15.32:0/1736747371 conn(0x7fe2ec0719c0 msgr2=0x7fe2ec071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 -- 172.21.15.32:0/1736747371 shutdown_connections 2023-10-16T08:17:28.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.129+0000 7fe2f133e640 1 -- 172.21.15.32:0/1736747371 wait complete. 2023-10-16T08:17:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 Processor -- start 2023-10-16T08:17:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 -- start start 2023-10-16T08:17:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2ec0d0210 0x7fe2ec148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2ec15cba0 0x7fe2ec149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec148b50 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec0c6770 con 0x7fe2ec0d0210 2023-10-16T08:17:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec156f40 con 0x7fe2ec15cba0 2023-10-16T08:17:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40634/0 (socket says 172.21.15.32:40634) 2023-10-16T08:17:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 -- 172.21.15.32:0/600820157 learned_addr learned my addr 172.21.15.32:0/600820157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:28.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2e3fff640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2ec0d0210 0x7fe2ec148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 -- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2ec15cba0 msgr2=0x7fe2ec149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eb7fe640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2ec15cba0 0x7fe2ec149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2ec15cba0 0x7fe2ec149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 -- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2ec0d0210 msgr2=0x7fe2ec148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2ec0d0210 0x7fe2ec148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 -- 172.21.15.32:0/600820157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4009b30 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2e3fff640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2ec0d0210 0x7fe2ec148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eb7fe640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2ec15cba0 0x7fe2ec149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:28.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2eaffd640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec148610 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c430 tx=0x7fe2dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0074e0 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc007640 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec1566a0 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc004a10 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.133+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec156c60 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.137+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec0b6030 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.137+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe2dc004b70 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.137+0000 7fe2e37fe640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2cc0664a0 0x7fe2cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.137+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe2dc083c70 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.137+0000 7fe2e3fff640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2cc0664a0 0x7fe2cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.137+0000 7fe2e3fff640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2cc0664a0 0x7fe2cc068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec150ac0 tx=0x7fe2d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.141+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe2dc050d20 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.297+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2ec04e170 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.297+0000 7fe2e37fe640 1 -- 172.21.15.32:0/600820157 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2dc054340 con 0x7fe2ec0b84e0 2023-10-16T08:17:28.306 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:28.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2cc0664a0 msgr2=0x7fe2cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2cc0664a0 0x7fe2cc068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec150ac0 tx=0x7fe2d4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 msgr2=0x7fe2ec148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec148610 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c430 tx=0x7fe2dc00c8f0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 shutdown_connections 2023-10-16T08:17:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2cc0664a0 0x7fe2cc068960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2ec15cba0 0x7fe2ec149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2ec0d0210 0x7fe2ec148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 --2- 172.21.15.32:0/600820157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2ec0b84e0 0x7fe2ec148610 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 >> 172.21.15.32:0/600820157 conn(0x7fe2ec0719c0 msgr2=0x7fe2ec153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 shutdown_connections 2023-10-16T08:17:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.301+0000 7fe2f133e640 1 -- 172.21.15.32:0/600820157 wait complete. 2023-10-16T08:17:28.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.465+0000 7f8c60c46640 1 Processor -- start 2023-10-16T08:17:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.465+0000 7f8c60c46640 1 -- start start 2023-10-16T08:17:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c60c46640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8c5c1590d0 0x7f8c5c06df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c60c46640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5c0c9600 con 0x7f8c5c0c8dd0 2023-10-16T08:17:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c60c46640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5c0c98e0 con 0x7f8c5c070a00 2023-10-16T08:17:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c60c46640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5c0c9bc0 con 0x7f8c5c1590d0 2023-10-16T08:17:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c53fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c5c0c8dd0 0x7f8c5c0702f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45804/0 (socket says 172.21.15.32:45804) 2023-10-16T08:17:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c53fff640 1 -- 172.21.15.32:0/348148424 learned_addr learned my addr 172.21.15.32:0/348148424 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/348148424 >> v2:172.21.15.186:3300/0 conn(0x7f8c5c1590d0 0x7f8c5c06df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313737806 0 0) 0x7f8c5c0c9600 con 0x7f8c5c0c8dd0 2023-10-16T08:17:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c48003530 con 0x7f8c5c0c8dd0 2023-10-16T08:17:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3737703784 0 0) 0x7f8c48003530 con 0x7f8c5c0c8dd0 2023-10-16T08:17:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 >> v2:172.21.15.186:3300/0 conn(0x7f8c5c1590d0 msgr2=0x7f8c5c06df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 --2- 172.21.15.32:0/348148424 >> v2:172.21.15.186:3300/0 conn(0x7f8c5c1590d0 0x7f8c5c06df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 >> v1:172.21.15.186:6789/0 conn(0x7f8c5c070a00 legacy=0x7f8c5c0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c0c9ea0 con 0x7f8c5c0c8dd0 2023-10-16T08:17:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f8c44001680 con 0x7f8c5c0c8dd0 2023-10-16T08:17:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 >> v1:172.21.15.32:6789/0 conn(0x7f8c5c0c8dd0 legacy=0x7f8c5c0702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c48003980 0x7f8c48003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c48004380 0x7f8c48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48005050 0x7f8c4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0c9bc0 con 0x7f8c48003980 2023-10-16T08:17:28.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4800bfb0 con 0x7f8c48005050 2023-10-16T08:17:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4800c170 con 0x7f8c48004380 2023-10-16T08:17:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c48003980 0x7f8c48003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c53fff640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c48004380 0x7f8c48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5ad9d640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48005050 0x7f8c4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 -- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c48004380 msgr2=0x7f8c48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c48004380 0x7f8c48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 -- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48005050 msgr2=0x7f8c4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48005050 0x7f8c4800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5a59c640 1 -- 172.21.15.32:0/348148424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48003530 con 0x7f8c48003980 2023-10-16T08:17:28.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5ad9d640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48005050 0x7f8c4800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:28.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.469+0000 7f8c5ad9d640 1 -- 172.21.15.32:0/348148424 reap_dead start 2023-10-16T08:17:28.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c48003980 0x7f8c48003d60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009ae0 tx=0x7f8c4c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402fbdb1fbdff23e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0a5640 con 0x7f8c48003980 2023-10-16T08:17:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c0a57a0 con 0x7f8c48003980 2023-10-16T08:17:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c59d9b640 1 -- 172.21.15.32:0/348148424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0a5b50 con 0x7f8c48003980 2023-10-16T08:17:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 -- 172.21.15.32:0/348148424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c48003980 msgr2=0x7f8c48003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c48003980 0x7f8c48003d60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009ae0 tx=0x7f8c4c01ca50 comp rx=0 tx=0).stop 2023-10-16T08:17:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 -- 172.21.15.32:0/348148424 shutdown_connections 2023-10-16T08:17:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 --2- 172.21.15.32:0/348148424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c48003980 0x7f8c48003d60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 -- 172.21.15.32:0/348148424 >> 172.21.15.32:0/348148424 conn(0x7f8c5c0b0da0 msgr2=0x7f8c5c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 -- 172.21.15.32:0/348148424 shutdown_connections 2023-10-16T08:17:28.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 -- 172.21.15.32:0/348148424 wait complete. 2023-10-16T08:17:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.473+0000 7f8c60c46640 1 Processor -- start 2023-10-16T08:17:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- start start 2023-10-16T08:17:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48003980 0x7f8c5c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c5c159630 0x7f8c5c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 0x7f8c5c0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0c8250 con 0x7f8c5c0c7860 2023-10-16T08:17:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c15a0f0 con 0x7f8c48003980 2023-10-16T08:17:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c1f2390 con 0x7f8c5c159630 2023-10-16T08:17:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 0x7f8c5c0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 0x7f8c5c0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40650/0 (socket says 172.21.15.32:40650) 2023-10-16T08:17:28.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 -- 172.21.15.32:0/2765454583 learned_addr learned my addr 172.21.15.32:0/2765454583 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48003980 0x7f8c5c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c53fff640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c5c159630 0x7f8c5c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 -- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c5c159630 msgr2=0x7f8c5c159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c5c159630 0x7f8c5c159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 -- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48003980 msgr2=0x7f8c5c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48003980 0x7f8c5c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 -- 172.21.15.32:0/2765454583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c1f2670 con 0x7f8c5c0c7860 2023-10-16T08:17:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c5ad9d640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 0x7f8c5c0c7d10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c070eb0 tx=0x7f8c54011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400fd10 con 0x7f8c5c0c7860 2023-10-16T08:17:28.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5400fe70 con 0x7f8c5c0c7860 2023-10-16T08:17:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54015610 con 0x7f8c5c0c7860 2023-10-16T08:17:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c1f2950 con 0x7f8c5c0c7860 2023-10-16T08:17:28.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c157f80 con 0x7f8c5c0c7860 2023-10-16T08:17:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.477+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c1f21e0 con 0x7f8c5c0c7860 2023-10-16T08:17:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.481+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8c54015770 con 0x7f8c5c0c7860 2023-10-16T08:17:28.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.481+0000 7f8c52ffd640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8c3c0664d0 0x7f8c3c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.481+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8c54087710 con 0x7f8c5c0c7860 2023-10-16T08:17:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.481+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8c3c0664d0 0x7f8c3c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.481+0000 7f8c5a59c640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8c3c0664d0 0x7f8c3c068990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0af000 tx=0x7f8c4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.485+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8c540547c0 con 0x7f8c5c0c7860 2023-10-16T08:17:28.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.641+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c5c04e170 con 0x7f8c5c0c7860 2023-10-16T08:17:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.641+0000 7f8c52ffd640 1 -- 172.21.15.32:0/2765454583 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c54057de0 con 0x7f8c5c0c7860 2023-10-16T08:17:28.649 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8c3c0664d0 msgr2=0x7f8c3c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8c3c0664d0 0x7f8c3c068990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0af000 tx=0x7f8c4c008040 comp rx=0 tx=0).stop 2023-10-16T08:17:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 msgr2=0x7f8c5c0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 0x7f8c5c0c7d10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c070eb0 tx=0x7f8c54011f70 comp rx=0 tx=0).stop 2023-10-16T08:17:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 shutdown_connections 2023-10-16T08:17:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c5c0c7860 0x7f8c5c0c7d10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8c3c0664d0 0x7f8c3c068990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8c5c159630 0x7f8c5c159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 --2- 172.21.15.32:0/2765454583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c48003980 0x7f8c5c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 >> 172.21.15.32:0/2765454583 conn(0x7f8c5c0b0da0 msgr2=0x7f8c5c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 shutdown_connections 2023-10-16T08:17:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.645+0000 7f8c60c46640 1 -- 172.21.15.32:0/2765454583 wait complete. 2023-10-16T08:17:28.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.809+0000 7fa6c3e17640 1 Processor -- start 2023-10-16T08:17:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.809+0000 7fa6c3e17640 1 -- start start 2023-10-16T08:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c3e17640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa6bc0d0210 0x7fa6bc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c3e17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc0d1670 con 0x7fa6bc0b84e0 2023-10-16T08:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c3e17640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc0d1950 con 0x7fa6bc15cba0 2023-10-16T08:17:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c3e17640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6bc0d1460 con 0x7fa6bc0d0210 2023-10-16T08:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6bc0b84e0 0x7fa6bc0b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45808/0 (socket says 172.21.15.32:45808) 2023-10-16T08:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 -- 172.21.15.32:0/2799244767 learned_addr learned my addr 172.21.15.32:0/2799244767 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c13b2640 1 --2- 172.21.15.32:0/2799244767 >> v2:172.21.15.186:3300/0 conn(0x7fa6bc0d0210 0x7fa6bc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452092634 0 0) 0x7fa6bc0d1670 con 0x7fa6bc0b84e0 2023-10-16T08:17:28.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a8003530 con 0x7fa6bc0b84e0 2023-10-16T08:17:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130598122 0 0) 0x7fa6bc0d1950 con 0x7fa6bc15cba0 2023-10-16T08:17:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6bc0d1670 con 0x7fa6bc15cba0 2023-10-16T08:17:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2309104772 0 0) 0x7fa6a8003530 con 0x7fa6bc0b84e0 2023-10-16T08:17:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 >> v2:172.21.15.186:3300/0 conn(0x7fa6bc0d0210 msgr2=0x7fa6bc0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 --2- 172.21.15.32:0/2799244767 >> v2:172.21.15.186:3300/0 conn(0x7fa6bc0d0210 0x7fa6bc0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 >> v1:172.21.15.186:6789/0 conn(0x7fa6bc15cba0 legacy=0x7fa6bc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0d1c30 con 0x7fa6bc0b84e0 2023-10-16T08:17:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fa6b4001a00 con 0x7fa6bc0b84e0 2023-10-16T08:17:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 >> v1:172.21.15.32:6789/0 conn(0x7fa6bc0b84e0 legacy=0x7fa6bc0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8003b10 0x7fa6a8003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a80047d0 0x7fa6a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a8005190 0x7fa6a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0d1460 con 0x7fa6a8005190 2023-10-16T08:17:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800c0c0 con 0x7fa6a80047d0 2023-10-16T08:17:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800c280 con 0x7fa6a8003b10 2023-10-16T08:17:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a8005190 0x7fa6a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c13b2640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a80047d0 0x7fa6a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c1bb3640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8003b10 0x7fa6a8003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 -- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8003b10 msgr2=0x7fa6a8003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8003b10 0x7fa6a8003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 -- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a80047d0 msgr2=0x7fa6a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a80047d0 0x7fa6a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.813+0000 7fa6c23b4640 1 -- 172.21.15.32:0/2799244767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8003530 con 0x7fa6a8005190 2023-10-16T08:17:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c23b4640 1 -- 172.21.15.32:0/2799244767 reap_dead start 2023-10-16T08:17:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a8005190 0x7fa6a800b990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0b8990 tx=0x7fa6b401d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e2eb8b902343a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40ac620 con 0x7fa6a8005190 2023-10-16T08:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40a8070 con 0x7fa6a8005190 2023-10-16T08:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c0bb1640 1 -- 172.21.15.32:0/2799244767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40ac9e0 con 0x7fa6a8005190 2023-10-16T08:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 -- 172.21.15.32:0/2799244767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a8005190 msgr2=0x7fa6a800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a8005190 0x7fa6a800b990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0b8990 tx=0x7fa6b401d280 comp rx=0 tx=0).stop 2023-10-16T08:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 -- 172.21.15.32:0/2799244767 shutdown_connections 2023-10-16T08:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/2799244767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a8005190 0x7fa6a800b990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 -- 172.21.15.32:0/2799244767 >> 172.21.15.32:0/2799244767 conn(0x7fa6bc0719c0 msgr2=0x7fa6bc071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 -- 172.21.15.32:0/2799244767 shutdown_connections 2023-10-16T08:17:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.817+0000 7fa6c3e17640 1 -- 172.21.15.32:0/2799244767 wait complete. 2023-10-16T08:17:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 Processor -- start 2023-10-16T08:17:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- start start 2023-10-16T08:17:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8005190 0x7fa6bc1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6bc1f4a70 0x7fa6bc0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 0x7fa6bc1e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0c79c0 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0c7120 con 0x7fa6bc1f4a70 2023-10-16T08:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0c7400 con 0x7fa6a8005190 2023-10-16T08:17:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 0x7fa6bc1e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 0x7fa6bc1e6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40672/0 (socket says 172.21.15.32:40672) 2023-10-16T08:17:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 -- 172.21.15.32:0/3206882880 learned_addr learned my addr 172.21.15.32:0/3206882880 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c1bb3640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8005190 0x7fa6bc1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c13b2640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6bc1f4a70 0x7fa6bc0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 -- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8005190 msgr2=0x7fa6bc1f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8005190 0x7fa6bc1f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 -- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6bc1f4a70 msgr2=0x7fa6bc0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6bc1f4a70 0x7fa6bc0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 -- 172.21.15.32:0/3206882880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0c76e0 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c23b4640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 0x7fa6bc1e6510 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4002240 tx=0x7fa6a8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40072c0 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4007420 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc15d120 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40077b0 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc0c6bc0 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.821+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc15cf80 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.825+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa6b40aa070 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.825+0000 7fa6b27fc640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa6a00664d0 0x7fa6a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.825+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa6b40b3070 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.825+0000 7fa6c1bb3640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa6a00664d0 0x7fa6a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.825+0000 7fa6c1bb3640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa6a00664d0 0x7fa6a0068990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005630 tx=0x7fa6b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:28.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.829+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa6bc15cf80 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.985+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa6bc15cf80 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6b27fc640 1 -- 172.21.15.32:0/3206882880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa6bc15cf80 con 0x7fa6bc1f4ef0 2023-10-16T08:17:28.998 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa6a00664d0 msgr2=0x7fa6a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa6a00664d0 0x7fa6a0068990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005630 tx=0x7fa6b8009040 comp rx=0 tx=0).stop 2023-10-16T08:17:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 msgr2=0x7fa6bc1e6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 0x7fa6bc1e6510 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4002240 tx=0x7fa6a8005800 comp rx=0 tx=0).stop 2023-10-16T08:17:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 shutdown_connections 2023-10-16T08:17:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6bc1f4ef0 0x7fa6bc1e6510 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6bc1f4a70 0x7fa6bc0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa6a00664d0 0x7fa6a0068990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 --2- 172.21.15.32:0/3206882880 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa6a8005190 0x7fa6bc1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 >> 172.21.15.32:0/3206882880 conn(0x7fa6bc0719c0 msgr2=0x7fa6bc0c8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 shutdown_connections 2023-10-16T08:17:29.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:28.989+0000 7fa6c3e17640 1 -- 172.21.15.32:0/3206882880 wait complete. 2023-10-16T08:17:29.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 Processor -- start 2023-10-16T08:17:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 -- start start 2023-10-16T08:17:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f04041597c0 0x7f04040ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0404159ba0 0x7f04040cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04040adc10 0x7f04040cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040723d0 con 0x7f04040adc10 2023-10-16T08:17:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040717c0 con 0x7f04041597c0 2023-10-16T08:17:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f040a7ab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0404071aa0 con 0x7f0404159ba0 2023-10-16T08:17:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04040adc10 0x7f04040cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04040adc10 0x7f04040cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40682/0 (socket says 172.21.15.32:40682) 2023-10-16T08:17:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 -- 172.21.15.32:0/2983896260 learned_addr learned my addr 172.21.15.32:0/2983896260 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0403fff640 1 --2- 172.21.15.32:0/2983896260 >> v2:172.21.15.186:3300/0 conn(0x7f04041597c0 0x7f04040ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f04037fe640 1 --2- 172.21.15.32:0/2983896260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0404159ba0 0x7f04040cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 -- 172.21.15.32:0/2983896260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0404159ba0 msgr2=0x7f04040cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 --2- 172.21.15.32:0/2983896260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0404159ba0 0x7f04040cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 -- 172.21.15.32:0/2983896260 >> v2:172.21.15.186:3300/0 conn(0x7f04041597c0 msgr2=0x7f04040ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 --2- 172.21.15.32:0/2983896260 >> v2:172.21.15.186:3300/0 conn(0x7f04041597c0 0x7f04040ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.153+0000 7f0408d48640 1 -- 172.21.15.32:0/2983896260 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404071d80 con 0x7f04040adc10 2023-10-16T08:17:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f0408d48640 1 --2- 172.21.15.32:0/2983896260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04040adc10 0x7f04040cb350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f04040cdaf0 tx=0x7f03fc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335ccfead73a12ea server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f0402ffd640 1 -- 172.21.15.32:0/2983896260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0a57c0 con 0x7f04040adc10 2023-10-16T08:17:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f0402ffd640 1 -- 172.21.15.32:0/2983896260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc0a5920 con 0x7f04040adc10 2023-10-16T08:17:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f0402ffd640 1 -- 172.21.15.32:0/2983896260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0a5cb0 con 0x7f04040adc10 2023-10-16T08:17:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 -- 172.21.15.32:0/2983896260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04040adc10 msgr2=0x7f04040cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 --2- 172.21.15.32:0/2983896260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04040adc10 0x7f04040cb350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f04040cdaf0 tx=0x7f03fc01c9a0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 -- 172.21.15.32:0/2983896260 shutdown_connections 2023-10-16T08:17:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 --2- 172.21.15.32:0/2983896260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04040adc10 0x7f04040cb350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 --2- 172.21.15.32:0/2983896260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0404159ba0 0x7f04040cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 --2- 172.21.15.32:0/2983896260 >> v2:172.21.15.186:3300/0 conn(0x7f04041597c0 0x7f04040ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 -- 172.21.15.32:0/2983896260 >> 172.21.15.32:0/2983896260 conn(0x7f040406c190 msgr2=0x7f040406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 -- 172.21.15.32:0/2983896260 shutdown_connections 2023-10-16T08:17:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.157+0000 7f040a7ab640 1 -- 172.21.15.32:0/2983896260 wait complete. 2023-10-16T08:17:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 Processor -- start 2023-10-16T08:17:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- start start 2023-10-16T08:17:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04040adc10 0x7f040415a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04041597c0 0x7f040414c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 0x7f040414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040415be50 con 0x7f0404159ba0 2023-10-16T08:17:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040415b2d0 con 0x7f04040adc10 2023-10-16T08:17:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040415b5b0 con 0x7f04041597c0 2023-10-16T08:17:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 0x7f040414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 0x7f040414a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40692/0 (socket says 172.21.15.32:40692) 2023-10-16T08:17:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 -- 172.21.15.32:0/1288006591 learned_addr learned my addr 172.21.15.32:0/1288006591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0403fff640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04040adc10 0x7f040415a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f04037fe640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04041597c0 0x7f040414c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 -- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04041597c0 msgr2=0x7f040414c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04041597c0 0x7f040414c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 -- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04040adc10 msgr2=0x7f040415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04040adc10 0x7f040415a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 -- 172.21.15.32:0/1288006591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc009b30 con 0x7f0404159ba0 2023-10-16T08:17:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0408d48640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 0x7f040414a240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f03fc01b050 tx=0x7f03fc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0b8670 con 0x7f0404159ba0 2023-10-16T08:17:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc0b87f0 con 0x7f0404159ba0 2023-10-16T08:17:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040415b890 con 0x7f0404159ba0 2023-10-16T08:17:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0b8ba0 con 0x7f0404159ba0 2023-10-16T08:17:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040415af10 con 0x7f0404159ba0 2023-10-16T08:17:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.161+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0404150fd0 con 0x7f0404159ba0 2023-10-16T08:17:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.165+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f03fc0b8d00 con 0x7f0404159ba0 2023-10-16T08:17:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.165+0000 7f0400ff9640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03f00664a0 0x7f03f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.165+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f03fc11f8d0 con 0x7f0404159ba0 2023-10-16T08:17:29.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.165+0000 7f0403fff640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03f00664a0 0x7f03f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.165+0000 7f0403fff640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03f00664a0 0x7f03f0068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f04040aed00 tx=0x7f03f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.169+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f03fc0ecb50 con 0x7f0404159ba0 2023-10-16T08:17:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.333+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0404074010 con 0x7f0404159ba0 2023-10-16T08:17:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f0400ff9640 1 -- 172.21.15.32:0/1288006591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f03fc0f0170 con 0x7f0404159ba0 2023-10-16T08:17:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03f00664a0 msgr2=0x7f03f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03f00664a0 0x7f03f0068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f04040aed00 tx=0x7f03f4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 msgr2=0x7f040414a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 0x7f040414a240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f03fc01b050 tx=0x7f03fc019740 comp rx=0 tx=0).stop 2023-10-16T08:17:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 shutdown_connections 2023-10-16T08:17:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0404159ba0 0x7f040414a240 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03f00664a0 0x7f03f0068960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f04041597c0 0x7f040414c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 --2- 172.21.15.32:0/1288006591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04040adc10 0x7f040415a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 >> 172.21.15.32:0/1288006591 conn(0x7f040406c190 msgr2=0x7f04040ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 shutdown_connections 2023-10-16T08:17:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.337+0000 7f040a7ab640 1 -- 172.21.15.32:0/1288006591 wait complete. 2023-10-16T08:17:29.364 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:17:29.364 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:29.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:29.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 Processor -- start 2023-10-16T08:17:29.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 -- start start 2023-10-16T08:17:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 0x7f8af00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af00d3010 0x7f8af00d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af00ce770 con 0x7f8af00d1400 2023-10-16T08:17:29.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af00d1890 con 0x7f8af015cb90 2023-10-16T08:17:29.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af7032640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00d0600 con 0x7f8af00d3010 2023-10-16T08:17:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af4dce640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 0x7f8af00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af00d3010 0x7f8af00d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af4dce640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 0x7f8af00cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:40936/0 (socket says 172.21.15.32:40936) 2023-10-16T08:17:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af4dce640 1 -- 172.21.15.32:0/3212435118 learned_addr learned my addr 172.21.15.32:0/3212435118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8aeffff640 1 -- 172.21.15.32:0/3212435118 >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 msgr2=0x7f8af00cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8aeffff640 1 --2- 172.21.15.32:0/3212435118 >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 0x7f8af00cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8aeffff640 1 -- 172.21.15.32:0/3212435118 >> v1:172.21.15.186:6789/0 conn(0x7f8af00d1400 legacy=0x7f8af00cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8aeffff640 1 -- 172.21.15.32:0/3212435118 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00c6450 con 0x7f8af00d3010 2023-10-16T08:17:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8aef7fe640 1 -- 172.21.15.32:0/3212435118 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386079081 0 0) 0x7f8af00ce770 con 0x7f8af00d1400 2023-10-16T08:17:29.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.553+0000 7f8af4dce640 1 --2- 172.21.15.32:0/3212435118 >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 0x7f8af00cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8aeffff640 1 --2- 172.21.15.32:0/3212435118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af00d3010 0x7f8af00d0ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0097c0 tx=0x7f8adc01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8358ad6f5864415b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8aef7fe640 1 -- 172.21.15.32:0/3212435118 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0ac620 con 0x7f8af00d3010 2023-10-16T08:17:29.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8aef7fe640 1 -- 172.21.15.32:0/3212435118 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc0ac780 con 0x7f8af00d3010 2023-10-16T08:17:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8aef7fe640 1 -- 172.21.15.32:0/3212435118 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0acb10 con 0x7f8af00d3010 2023-10-16T08:17:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 -- 172.21.15.32:0/3212435118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af00d3010 msgr2=0x7f8af00d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 --2- 172.21.15.32:0/3212435118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af00d3010 0x7f8af00d0ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0097c0 tx=0x7f8adc01c3c0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 -- 172.21.15.32:0/3212435118 shutdown_connections 2023-10-16T08:17:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 --2- 172.21.15.32:0/3212435118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af00d3010 0x7f8af00d0ec0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 --2- 172.21.15.32:0/3212435118 >> v2:172.21.15.186:3300/0 conn(0x7f8af015cb90 0x7f8af00cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 -- 172.21.15.32:0/3212435118 >> 172.21.15.32:0/3212435118 conn(0x7f8af009bc50 msgr2=0x7f8af00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 -- 172.21.15.32:0/3212435118 shutdown_connections 2023-10-16T08:17:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.557+0000 7f8af7032640 1 -- 172.21.15.32:0/3212435118 wait complete. 2023-10-16T08:17:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 Processor -- start 2023-10-16T08:17:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 -- start start 2023-10-16T08:17:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af00d1400 0x7f8af0152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 0x7f8af006b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af015cb90 0x7f8af006b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b6960 con 0x7f8af00d3010 2023-10-16T08:17:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b84e0 con 0x7f8af00d1400 2023-10-16T08:17:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af7032640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b7ed0 con 0x7f8af015cb90 2023-10-16T08:17:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 0x7f8af006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 0x7f8af006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40706/0 (socket says 172.21.15.32:40706) 2023-10-16T08:17:29.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 -- 172.21.15.32:0/410571575 learned_addr learned my addr 172.21.15.32:0/410571575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af55cf640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af015cb90 0x7f8af006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af4dce640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af00d1400 0x7f8af0152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 -- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af015cb90 msgr2=0x7f8af006b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af015cb90 0x7f8af006b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 -- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af00d1400 msgr2=0x7f8af0152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af00d1400 0x7f8af0152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 -- 172.21.15.32:0/410571575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4009480 con 0x7f8af00d3010 2023-10-16T08:17:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af55cf640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af015cb90 0x7f8af006b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8af4dce640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af00d1400 0x7f8af0152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:29.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aeffff640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 0x7f8af006b110 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8adc018fd0 tx=0x7f8adc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.561+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0b5400 con 0x7f8af00d3010 2023-10-16T08:17:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc0b5560 con 0x7f8af00d3010 2023-10-16T08:17:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0b58f0 con 0x7f8af00d3010 2023-10-16T08:17:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc009420 con 0x7f8af00d3010 2023-10-16T08:17:29.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af00b57d0 con 0x7f8af00d3010 2023-10-16T08:17:29.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af00b7d30 con 0x7f8af00d3010 2023-10-16T08:17:29.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8adc0b3040 con 0x7f8af00d3010 2023-10-16T08:17:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8aed7fa640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8ad40664a0 0x7f8ad4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8adc11df90 con 0x7f8af00d3010 2023-10-16T08:17:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8af4dce640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8ad40664a0 0x7f8ad4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.565+0000 7f8af4dce640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8ad40664a0 0x7f8ad4068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8af006be40 tx=0x7f8ae4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.569+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8af00b7d30 con 0x7f8af00d3010 2023-10-16T08:17:29.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.725+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8af00b7d30 con 0x7f8af00d3010 2023-10-16T08:17:29.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8aed7fa640 1 -- 172.21.15.32:0/410571575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f8af00b7d30 con 0x7f8af00d3010 2023-10-16T08:17:29.734 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:29.734 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8ad40664a0 msgr2=0x7f8ad4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8ad40664a0 0x7f8ad4068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8af006be40 tx=0x7f8ae4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 msgr2=0x7f8af006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 0x7f8af006b110 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8adc018fd0 tx=0x7f8adc019160 comp rx=0 tx=0).stop 2023-10-16T08:17:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 shutdown_connections 2023-10-16T08:17:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8ad40664a0 0x7f8ad4068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8af015cb90 0x7f8af006b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af00d3010 0x7f8af006b110 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 --2- 172.21.15.32:0/410571575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af00d1400 0x7f8af0152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 >> 172.21.15.32:0/410571575 conn(0x7f8af009bc50 msgr2=0x7f8af0156040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 shutdown_connections 2023-10-16T08:17:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.729+0000 7f8af7032640 1 -- 172.21.15.32:0/410571575 wait complete. 2023-10-16T08:17:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:29.756 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:29.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 Processor -- start 2023-10-16T08:17:29.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 -- start start 2023-10-16T08:17:29.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e5415ce10 0x7f5e54161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5e540cd3e0 0x7f5e540ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5406e2f0 0x7f5e5406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5406eca0 con 0x7f5e5415ce10 2023-10-16T08:17:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e540cdc30 con 0x7f5e540cd3e0 2023-10-16T08:17:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e59c68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540c9750 con 0x7f5e5406e2f0 2023-10-16T08:17:29.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e5415ce10 0x7f5e54161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e5415ce10 0x7f5e54161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40718/0 (socket says 172.21.15.32:40718) 2023-10-16T08:17:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 -- 172.21.15.32:0/2883043455 learned_addr learned my addr 172.21.15.32:0/2883043455 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e53fff640 1 --2- 172.21.15.32:0/2883043455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5406e2f0 0x7f5e5406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e52ffd640 1 --2- 172.21.15.32:0/2883043455 >> v2:172.21.15.186:3300/0 conn(0x7f5e540cd3e0 0x7f5e540ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 -- 172.21.15.32:0/2883043455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5406e2f0 msgr2=0x7f5e5406e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 --2- 172.21.15.32:0/2883043455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5406e2f0 0x7f5e5406e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 -- 172.21.15.32:0/2883043455 >> v2:172.21.15.186:3300/0 conn(0x7f5e540cd3e0 msgr2=0x7f5e540ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 --2- 172.21.15.32:0/2883043455 >> v2:172.21.15.186:3300/0 conn(0x7f5e540cd3e0 0x7f5e540ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.897+0000 7f5e537fe640 1 -- 172.21.15.32:0/2883043455 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540c99f0 con 0x7f5e5415ce10 2023-10-16T08:17:29.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e537fe640 1 --2- 172.21.15.32:0/2883043455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5415ce10 0x7f5e54161390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5e540d0020 tx=0x7f5e4401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80caec2511b27c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e527fc640 1 -- 172.21.15.32:0/2883043455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440aa620 con 0x7f5e5415ce10 2023-10-16T08:17:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e527fc640 1 -- 172.21.15.32:0/2883043455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e440aa780 con 0x7f5e5415ce10 2023-10-16T08:17:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e527fc640 1 -- 172.21.15.32:0/2883043455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440aab10 con 0x7f5e5415ce10 2023-10-16T08:17:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 -- 172.21.15.32:0/2883043455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5415ce10 msgr2=0x7f5e54161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 --2- 172.21.15.32:0/2883043455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5415ce10 0x7f5e54161390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5e540d0020 tx=0x7f5e4401c500 comp rx=0 tx=0).stop 2023-10-16T08:17:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 -- 172.21.15.32:0/2883043455 shutdown_connections 2023-10-16T08:17:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 --2- 172.21.15.32:0/2883043455 >> v2:172.21.15.186:3300/0 conn(0x7f5e540cd3e0 0x7f5e540ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 --2- 172.21.15.32:0/2883043455 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e5406e2f0 0x7f5e5406e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 --2- 172.21.15.32:0/2883043455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5415ce10 0x7f5e54161390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 -- 172.21.15.32:0/2883043455 >> 172.21.15.32:0/2883043455 conn(0x7f5e540cef20 msgr2=0x7f5e540cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:29.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 -- 172.21.15.32:0/2883043455 shutdown_connections 2023-10-16T08:17:29.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 -- 172.21.15.32:0/2883043455 wait complete. 2023-10-16T08:17:29.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.901+0000 7f5e59c68640 1 Processor -- start 2023-10-16T08:17:29.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- start start 2023-10-16T08:17:29.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 0x7f5e541473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e540cd3e0 0x7f5e540afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e540b0040 0x7f5e540c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e541478e0 con 0x7f5e5406e2f0 2023-10-16T08:17:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540b0550 con 0x7f5e540cd3e0 2023-10-16T08:17:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540b0830 con 0x7f5e540b0040 2023-10-16T08:17:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 0x7f5e541473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 0x7f5e541473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40720/0 (socket says 172.21.15.32:40720) 2023-10-16T08:17:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 -- 172.21.15.32:0/967958538 learned_addr learned my addr 172.21.15.32:0/967958538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e53fff640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e540b0040 0x7f5e540c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e52ffd640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e540cd3e0 0x7f5e540afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 -- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e540b0040 msgr2=0x7f5e540c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e540b0040 0x7f5e540c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 -- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e540cd3e0 msgr2=0x7f5e540afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e540cd3e0 0x7f5e540afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 -- 172.21.15.32:0/967958538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440094d0 con 0x7f5e5406e2f0 2023-10-16T08:17:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e52ffd640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e540cd3e0 0x7f5e540afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e537fe640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 0x7f5e541473a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5e4401cda0 tx=0x7f5e4401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440aae80 con 0x7f5e5406e2f0 2023-10-16T08:17:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e440bd3f0 con 0x7f5e5406e2f0 2023-10-16T08:17:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e540c2950 con 0x7f5e5406e2f0 2023-10-16T08:17:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440bd780 con 0x7f5e5406e2f0 2023-10-16T08:17:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e540c2f10 con 0x7f5e5406e2f0 2023-10-16T08:17:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.905+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e540c3100 con 0x7f5e5406e2f0 2023-10-16T08:17:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.909+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5e440b33f0 con 0x7f5e5406e2f0 2023-10-16T08:17:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.909+0000 7f5e3bfff640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e400664a0 0x7f5e40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.909+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5e4411e7c0 con 0x7f5e5406e2f0 2023-10-16T08:17:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.909+0000 7f5e52ffd640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e400664a0 0x7f5e40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.909+0000 7f5e52ffd640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e400664a0 0x7f5e40068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e5415e2b0 tx=0x7f5e48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:29.913+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5e440eb890 con 0x7f5e5406e2f0 2023-10-16T08:17:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.069+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e5404e170 con 0x7f5e5406e2f0 2023-10-16T08:17:30.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.069+0000 7f5e3bfff640 1 -- 172.21.15.32:0/967958538 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e440eeeb0 con 0x7f5e5406e2f0 2023-10-16T08:17:30.078 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:30.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e400664a0 msgr2=0x7f5e40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e400664a0 0x7f5e40068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e5415e2b0 tx=0x7f5e48008040 comp rx=0 tx=0).stop 2023-10-16T08:17:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 msgr2=0x7f5e541473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 0x7f5e541473a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5e4401cda0 tx=0x7f5e4401ad40 comp rx=0 tx=0).stop 2023-10-16T08:17:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 shutdown_connections 2023-10-16T08:17:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5e400664a0 0x7f5e40068960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5e540b0040 0x7f5e540c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e540cd3e0 0x7f5e540afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 --2- 172.21.15.32:0/967958538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5406e2f0 0x7f5e541473a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 >> 172.21.15.32:0/967958538 conn(0x7f5e540cef20 msgr2=0x7f5e540c8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:30.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 shutdown_connections 2023-10-16T08:17:30.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.073+0000 7f5e59c68640 1 -- 172.21.15.32:0/967958538 wait complete. 2023-10-16T08:17:30.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:30.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 Processor -- start 2023-10-16T08:17:30.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 -- start start 2023-10-16T08:17:30.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1bb00d0210 0x7f1bb00d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1bb00b84e0 0x7f1bb00b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb00d1670 con 0x7f1bb015cba0 2023-10-16T08:17:30.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00d1460 con 0x7f1bb00b84e0 2023-10-16T08:17:30.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb6c6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00d1950 con 0x7f1bb00d0210 2023-10-16T08:17:30.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1bb00b84e0 0x7f1bb00b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1bb00b84e0 0x7f1bb00b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40732/0 (socket says 172.21.15.32:40732) 2023-10-16T08:17:30.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 -- 172.21.15.32:0/2951261583 learned_addr learned my addr 172.21.15.32:0/2951261583 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:30.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 -- 172.21.15.32:0/2951261583 >> v2:172.21.15.186:3300/0 conn(0x7f1bb00d0210 msgr2=0x7f1bb00d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1baffff640 1 --2- 172.21.15.32:0/2951261583 >> v2:172.21.15.186:3300/0 conn(0x7f1bb00d0210 0x7f1bb00d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 --2- 172.21.15.32:0/2951261583 >> v2:172.21.15.186:3300/0 conn(0x7f1bb00d0210 0x7f1bb00d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 -- 172.21.15.32:0/2951261583 >> v1:172.21.15.186:6789/0 conn(0x7f1bb015cba0 legacy=0x7f1bb00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1bb5208640 1 -- 172.21.15.32:0/2951261583 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00d1c30 con 0x7f1bb00b84e0 2023-10-16T08:17:30.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.241+0000 7f1baffff640 1 --2- 172.21.15.32:0/2951261583 >> v2:172.21.15.186:3300/0 conn(0x7f1bb00d0210 0x7f1bb00d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb5208640 1 --2- 172.21.15.32:0/2951261583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00b84e0 0x7f1bb00b5cf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cf010 tx=0x7f1ba801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb76e6a40877ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1baf7fe640 1 -- 172.21.15.32:0/2951261583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80a57c0 con 0x7f1bb00b84e0 2023-10-16T08:17:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1baf7fe640 1 -- 172.21.15.32:0/2951261583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80a5920 con 0x7f1bb00b84e0 2023-10-16T08:17:30.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1baf7fe640 1 -- 172.21.15.32:0/2951261583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80a5cb0 con 0x7f1bb00b84e0 2023-10-16T08:17:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/2951261583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00b84e0 msgr2=0x7f1bb00b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/2951261583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00b84e0 0x7f1bb00b5cf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cf010 tx=0x7f1ba801c900 comp rx=0 tx=0).stop 2023-10-16T08:17:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/2951261583 shutdown_connections 2023-10-16T08:17:30.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/2951261583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00b84e0 0x7f1bb00b5cf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/2951261583 >> v2:172.21.15.186:3300/0 conn(0x7f1bb00d0210 0x7f1bb00d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/2951261583 >> 172.21.15.32:0/2951261583 conn(0x7f1bb00719c0 msgr2=0x7f1bb0071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:30.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/2951261583 shutdown_connections 2023-10-16T08:17:30.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/2951261583 wait complete. 2023-10-16T08:17:30.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 Processor -- start 2023-10-16T08:17:30.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.245+0000 7f1bb6c6b640 1 -- start start 2023-10-16T08:17:30.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb00b84e0 0x7f1bb01591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 0x7f1bb01f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bb015cba0 0x7f1bb014e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb01f5f40 con 0x7f1bb00d0210 2023-10-16T08:17:30.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb01560d0 con 0x7f1bb00b84e0 2023-10-16T08:17:30.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0159710 con 0x7f1bb015cba0 2023-10-16T08:17:30.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 0x7f1bb01f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 0x7f1bb01f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40742/0 (socket says 172.21.15.32:40742) 2023-10-16T08:17:30.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 -- 172.21.15.32:0/81009167 learned_addr learned my addr 172.21.15.32:0/81009167 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:30.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb4a07640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb00b84e0 0x7f1bb01591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb5208640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bb015cba0 0x7f1bb014e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 -- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bb015cba0 msgr2=0x7f1bb014e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bb015cba0 0x7f1bb014e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 -- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb00b84e0 msgr2=0x7f1bb01591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb00b84e0 0x7f1bb01591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 -- 172.21.15.32:0/81009167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009b30 con 0x7f1bb00d0210 2023-10-16T08:17:30.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1baffff640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 0x7f1bb01f4d30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c7f0 tx=0x7f1b9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00cea0 con 0x7f1bb00d0210 2023-10-16T08:17:30.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c007a30 con 0x7f1bb00d0210 2023-10-16T08:17:30.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb01f56a0 con 0x7f1bb00d0210 2023-10-16T08:17:30.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c007de0 con 0x7f1bb00d0210 2023-10-16T08:17:30.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb01f5c60 con 0x7f1bb00d0210 2023-10-16T08:17:30.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.249+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb004e170 con 0x7f1bb00d0210 2023-10-16T08:17:30.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.253+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b9c0113f0 con 0x7f1bb00d0210 2023-10-16T08:17:30.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.253+0000 7f1bad7fa640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1b940664a0 0x7f1b94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.253+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1b9c084410 con 0x7f1bb00d0210 2023-10-16T08:17:30.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.253+0000 7f1bb4a07640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1b940664a0 0x7f1b94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.253+0000 7f1bb4a07640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1b940664a0 0x7f1b94068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40098e0 tx=0x7f1ba4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.257+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b9c0514e0 con 0x7f1bb00d0210 2023-10-16T08:17:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bb0155ea0 con 0x7f1bb00d0210 2023-10-16T08:17:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bad7fa640 1 -- 172.21.15.32:0/81009167 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b9c054b00 con 0x7f1bb00d0210 2023-10-16T08:17:30.419 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:30.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1b940664a0 msgr2=0x7f1b94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1b940664a0 0x7f1b94068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40098e0 tx=0x7f1ba4009060 comp rx=0 tx=0).stop 2023-10-16T08:17:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 msgr2=0x7f1bb01f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 0x7f1bb01f4d30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c7f0 tx=0x7f1b9c00ccb0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 shutdown_connections 2023-10-16T08:17:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1b940664a0 0x7f1b94068960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1bb015cba0 0x7f1bb014e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb00d0210 0x7f1bb01f4d30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 --2- 172.21.15.32:0/81009167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb00b84e0 0x7f1bb01591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 >> 172.21.15.32:0/81009167 conn(0x7f1bb00719c0 msgr2=0x7f1bb0072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:30.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 shutdown_connections 2023-10-16T08:17:30.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.413+0000 7f1bb6c6b640 1 -- 172.21.15.32:0/81009167 wait complete. 2023-10-16T08:17:30.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:30.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.577+0000 7fd10a794640 1 Processor -- start 2023-10-16T08:17:30.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.577+0000 7fd10a794640 1 -- start start 2023-10-16T08:17:30.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd10a794640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd104159900 0x7fd10406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd10a794640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd10406a5a0 0x7fd1040c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd10a794640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1040b4390 con 0x7fd1040b3f00 2023-10-16T08:17:30.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd10a794640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040cd740 con 0x7fd10406a5a0 2023-10-16T08:17:30.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd10a794640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040b4600 con 0x7fd104159900 2023-10-16T08:17:30.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd10406a5a0 0x7fd1040c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd10406a5a0 0x7fd1040c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40750/0 (socket says 172.21.15.32:40750) 2023-10-16T08:17:30.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 -- 172.21.15.32:0/3449804469 learned_addr learned my addr 172.21.15.32:0/3449804469 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:30.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd103fff640 1 --2- 172.21.15.32:0/3449804469 >> v2:172.21.15.186:3300/0 conn(0x7fd104159900 0x7fd10406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 -- 172.21.15.32:0/3449804469 >> v2:172.21.15.186:3300/0 conn(0x7fd104159900 msgr2=0x7fd10406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:30.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 --2- 172.21.15.32:0/3449804469 >> v2:172.21.15.186:3300/0 conn(0x7fd104159900 0x7fd10406e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 -- 172.21.15.32:0/3449804469 >> v1:172.21.15.186:6789/0 conn(0x7fd1040b3f00 legacy=0x7fd104069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:30.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.581+0000 7fd108d31640 1 -- 172.21.15.32:0/3449804469 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040c8dd0 con 0x7fd10406a5a0 2023-10-16T08:17:30.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd108d31640 1 --2- 172.21.15.32:0/3449804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd10406a5a0 0x7fd1040c9b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd10406ab70 tx=0x7fd0fc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262738909948f5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd102ffd640 1 -- 172.21.15.32:0/3449804469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0a5820 con 0x7fd10406a5a0 2023-10-16T08:17:30.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd102ffd640 1 -- 172.21.15.32:0/3449804469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc0a5980 con 0x7fd10406a5a0 2023-10-16T08:17:30.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd102ffd640 1 -- 172.21.15.32:0/3449804469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0a5d10 con 0x7fd10406a5a0 2023-10-16T08:17:30.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 -- 172.21.15.32:0/3449804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd10406a5a0 msgr2=0x7fd1040c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 --2- 172.21.15.32:0/3449804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd10406a5a0 0x7fd1040c9b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd10406ab70 tx=0x7fd0fc01c960 comp rx=0 tx=0).stop 2023-10-16T08:17:30.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 -- 172.21.15.32:0/3449804469 shutdown_connections 2023-10-16T08:17:30.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 --2- 172.21.15.32:0/3449804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd10406a5a0 0x7fd1040c9b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 --2- 172.21.15.32:0/3449804469 >> v2:172.21.15.186:3300/0 conn(0x7fd104159900 0x7fd10406e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 -- 172.21.15.32:0/3449804469 >> 172.21.15.32:0/3449804469 conn(0x7fd1040b0da0 msgr2=0x7fd1040ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 -- 172.21.15.32:0/3449804469 shutdown_connections 2023-10-16T08:17:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 -- 172.21.15.32:0/3449804469 wait complete. 2023-10-16T08:17:30.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 Processor -- start 2023-10-16T08:17:30.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.585+0000 7fd10a794640 1 -- start start 2023-10-16T08:17:30.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10406a5a0 0x7fd104151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1040b3f00 0x7fd1040abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 0x7fd1040ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040c4f70 con 0x7fd104159900 2023-10-16T08:17:30.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd104152ab0 con 0x7fd10406a5a0 2023-10-16T08:17:30.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10415cc60 con 0x7fd1040b3f00 2023-10-16T08:17:30.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 0x7fd1040ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 0x7fd1040ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40766/0 (socket says 172.21.15.32:40766) 2023-10-16T08:17:30.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 -- 172.21.15.32:0/2092788710 learned_addr learned my addr 172.21.15.32:0/2092788710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:30.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd103fff640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10406a5a0 0x7fd104151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd1037fe640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1040b3f00 0x7fd1040abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 -- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1040b3f00 msgr2=0x7fd1040abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1040b3f00 0x7fd1040abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 -- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10406a5a0 msgr2=0x7fd104151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10406a5a0 0x7fd104151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 -- 172.21.15.32:0/2092788710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc009b30 con 0x7fd104159900 2023-10-16T08:17:30.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd108d31640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 0x7fd1040ac350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0af040 tx=0x7fd0fc019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0b8670 con 0x7fd104159900 2023-10-16T08:17:30.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc0b87f0 con 0x7fd104159900 2023-10-16T08:17:30.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0b8ba0 con 0x7fd104159900 2023-10-16T08:17:30.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd104156b30 con 0x7fd104159900 2023-10-16T08:17:30.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041570f0 con 0x7fd104159900 2023-10-16T08:17:30.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd104156990 con 0x7fd104159900 2023-10-16T08:17:30.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.589+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0fc0b8d00 con 0x7fd104159900 2023-10-16T08:17:30.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.593+0000 7fd100ff9640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0f00664d0 0x7fd0f0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.593+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd0fc11f7a0 con 0x7fd104159900 2023-10-16T08:17:30.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.593+0000 7fd103fff640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0f00664d0 0x7fd0f0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.593+0000 7fd103fff640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0f00664d0 0x7fd0f0068990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40079c0 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.597+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd104156990 con 0x7fd104159900 2023-10-16T08:17:30.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.749+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd104156990 con 0x7fd104159900 2023-10-16T08:17:30.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.749+0000 7fd100ff9640 1 -- 172.21.15.32:0/2092788710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd104156990 con 0x7fd104159900 2023-10-16T08:17:30.758 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0f00664d0 msgr2=0x7fd0f0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0f00664d0 0x7fd0f0068990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40079c0 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:30.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 msgr2=0x7fd1040ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 0x7fd1040ac350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0af040 tx=0x7fd0fc019700 comp rx=0 tx=0).stop 2023-10-16T08:17:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 shutdown_connections 2023-10-16T08:17:30.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd104159900 0x7fd1040ac350 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0f00664d0 0x7fd0f0068990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1040b3f00 0x7fd1040abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 --2- 172.21.15.32:0/2092788710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd10406a5a0 0x7fd104151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 >> 172.21.15.32:0/2092788710 conn(0x7fd1040b0da0 msgr2=0x7fd1040ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 shutdown_connections 2023-10-16T08:17:30.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.753+0000 7fd10a794640 1 -- 172.21.15.32:0/2092788710 wait complete. 2023-10-16T08:17:30.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 Processor -- start 2023-10-16T08:17:30.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 -- start start 2023-10-16T08:17:30.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7af40d0210 0x7f7af40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40b84e0 0x7f7af40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af40d1670 con 0x7f7af415cba0 2023-10-16T08:17:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af40d1460 con 0x7f7af40d0210 2023-10-16T08:17:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af9291640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40d1950 con 0x7f7af40b84e0 2023-10-16T08:17:30.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7af415cba0 0x7f7af40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45812/0 (socket says 172.21.15.32:45812) 2023-10-16T08:17:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af2ffd640 1 -- 172.21.15.32:0/4015285987 learned_addr learned my addr 172.21.15.32:0/4015285987 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af27fc640 1 --2- 172.21.15.32:0/4015285987 >> v2:172.21.15.32:3300/0 conn(0x7f7af40d0210 0x7f7af40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af37fe640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40b84e0 0x7f7af40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af1ffb640 1 -- 172.21.15.32:0/4015285987 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782761713 0 0) 0x7f7af40d1670 con 0x7f7af415cba0 2023-10-16T08:17:30.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af1ffb640 1 -- 172.21.15.32:0/4015285987 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7adc003530 con 0x7f7af415cba0 2023-10-16T08:17:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af27fc640 1 -- 172.21.15.32:0/4015285987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40b84e0 msgr2=0x7f7af40b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af27fc640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40b84e0 0x7f7af40b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af27fc640 1 -- 172.21.15.32:0/4015285987 >> v1:172.21.15.32:6789/0 conn(0x7f7af415cba0 legacy=0x7f7af40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af27fc640 1 -- 172.21.15.32:0/4015285987 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40d1c30 con 0x7f7af40d0210 2023-10-16T08:17:30.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.921+0000 7f7af37fe640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40b84e0 0x7f7af40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af27fc640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40d0210 0x7f7af40d0f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097c0 tx=0x7f7ae801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923a0bb878c92e5e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af1ffb640 1 -- 172.21.15.32:0/4015285987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80ab680 con 0x7f7af40d0210 2023-10-16T08:17:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af1ffb640 1 -- 172.21.15.32:0/4015285987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae80ab7e0 con 0x7f7af40d0210 2023-10-16T08:17:30.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af1ffb640 1 -- 172.21.15.32:0/4015285987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80abb70 con 0x7f7af40d0210 2023-10-16T08:17:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 -- 172.21.15.32:0/4015285987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40d0210 msgr2=0x7f7af40d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40d0210 0x7f7af40d0f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097c0 tx=0x7f7ae801c480 comp rx=0 tx=0).stop 2023-10-16T08:17:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 -- 172.21.15.32:0/4015285987 shutdown_connections 2023-10-16T08:17:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af40b84e0 0x7f7af40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 --2- 172.21.15.32:0/4015285987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40d0210 0x7f7af40d0f20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 -- 172.21.15.32:0/4015285987 >> 172.21.15.32:0/4015285987 conn(0x7f7af40719c0 msgr2=0x7f7af4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 -- 172.21.15.32:0/4015285987 shutdown_connections 2023-10-16T08:17:30.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 -- 172.21.15.32:0/4015285987 wait complete. 2023-10-16T08:17:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.925+0000 7f7af9291640 1 Processor -- start 2023-10-16T08:17:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- start start 2023-10-16T08:17:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40b84e0 0x7f7af414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 0x7f7af40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af415cba0 0x7f7af406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af40c7fc0 con 0x7f7af40b84e0 2023-10-16T08:17:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af414e610 con 0x7f7af40d0210 2023-10-16T08:17:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af414f190 con 0x7f7af415cba0 2023-10-16T08:17:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af27fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 0x7f7af40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af37fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af415cba0 0x7f7af406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40b84e0 0x7f7af414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af27fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 0x7f7af40c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41018/0 (socket says 172.21.15.32:41018) 2023-10-16T08:17:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af27fc640 1 -- 172.21.15.32:0/3914115038 learned_addr learned my addr 172.21.15.32:0/3914115038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 -- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af415cba0 msgr2=0x7f7af406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af415cba0 0x7f7af406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 -- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 msgr2=0x7f7af40c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 0x7f7af40c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 -- 172.21.15.32:0/3914115038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8009420 con 0x7f7af40b84e0 2023-10-16T08:17:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af27fc640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 0x7f7af40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af2ffd640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40b84e0 0x7f7af414dcd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c390 tx=0x7f7ae400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4005cf0 con 0x7f7af40b84e0 2023-10-16T08:17:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4005e50 con 0x7f7af40b84e0 2023-10-16T08:17:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af414e860 con 0x7f7af40b84e0 2023-10-16T08:17:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40072d0 con 0x7f7af40b84e0 2023-10-16T08:17:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af414ee20 con 0x7f7af40b84e0 2023-10-16T08:17:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.929+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af404e170 con 0x7f7af40b84e0 2023-10-16T08:17:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.933+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7ae4007430 con 0x7f7af40b84e0 2023-10-16T08:17:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.933+0000 7f7adb7fe640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ae00664a0 0x7f7ae0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.933+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ae40836a0 con 0x7f7af40b84e0 2023-10-16T08:17:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.933+0000 7f7af27fc640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ae00664a0 0x7f7ae0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.933+0000 7f7af27fc640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ae00664a0 0x7f7ae0068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097c0 tx=0x7f7ae80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:30.937+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7ae4050800 con 0x7f7af40b84e0 2023-10-16T08:17:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.093+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7af4159df0 con 0x7f7af40b84e0 2023-10-16T08:17:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7adb7fe640 1 -- 172.21.15.32:0/3914115038 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ae4053e20 con 0x7f7af40b84e0 2023-10-16T08:17:31.103 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ae00664a0 msgr2=0x7f7ae0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ae00664a0 0x7f7ae0068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097c0 tx=0x7f7ae80a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40b84e0 msgr2=0x7f7af414dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40b84e0 0x7f7af414dcd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c390 tx=0x7f7ae400c850 comp rx=0 tx=0).stop 2023-10-16T08:17:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 shutdown_connections 2023-10-16T08:17:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ae00664a0 0x7f7ae0068960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7af415cba0 0x7f7af406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7af40d0210 0x7f7af40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 --2- 172.21.15.32:0/3914115038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af40b84e0 0x7f7af414dcd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 >> 172.21.15.32:0/3914115038 conn(0x7f7af40719c0 msgr2=0x7f7af4071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 shutdown_connections 2023-10-16T08:17:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.097+0000 7f7af9291640 1 -- 172.21.15.32:0/3914115038 wait complete. 2023-10-16T08:17:31.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96fda91640 1 Processor -- start 2023-10-16T08:17:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96fda91640 1 -- start start 2023-10-16T08:17:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96fda91640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f96f80b07f0 0x7f96f8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96fda91640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8071020 con 0x7f96f81591e0 2023-10-16T08:17:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96fda91640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f80c8e50 con 0x7f96f8071c70 2023-10-16T08:17:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96fda91640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f80c9130 con 0x7f96f80b07f0 2023-10-16T08:17:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96f81591e0 0x7f96f80c9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45828/0 (socket says 172.21.15.32:45828) 2023-10-16T08:17:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 -- 172.21.15.32:0/1746563245 learned_addr learned my addr 172.21.15.32:0/1746563245 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062897644 0 0) 0x7f96f8071020 con 0x7f96f81591e0 2023-10-16T08:17:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f7fff640 1 --2- 172.21.15.32:0/1746563245 >> v2:172.21.15.186:3300/0 conn(0x7f96f80b07f0 0x7f96f8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e0003450 con 0x7f96f81591e0 2023-10-16T08:17:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695386468 0 0) 0x7f96e0003450 con 0x7f96f81591e0 2023-10-16T08:17:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 >> v2:172.21.15.186:3300/0 conn(0x7f96f80b07f0 msgr2=0x7f96f8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 --2- 172.21.15.32:0/1746563245 >> v2:172.21.15.186:3300/0 conn(0x7f96f80b07f0 0x7f96f8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 >> v1:172.21.15.186:6789/0 conn(0x7f96f8071c70 legacy=0x7f96f80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80c9410 con 0x7f96f81591e0 2023-10-16T08:17:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f96e8001680 con 0x7f96f81591e0 2023-10-16T08:17:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 >> v1:172.21.15.32:6789/0 conn(0x7f96f81591e0 legacy=0x7f96f80c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e0003a70 0x7f96e0003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0008860 0x7f96e000ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e000b490 0x7f96e000d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80c9130 con 0x7f96e0003a70 2023-10-16T08:17:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e000e230 con 0x7f96e0008860 2023-10-16T08:17:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e000e3f0 con 0x7f96e000b490 2023-10-16T08:17:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e0003a70 0x7f96e0003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f6ffd640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0008860 0x7f96e000ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f7fff640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e000b490 0x7f96e000d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 -- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e000b490 msgr2=0x7f96e000d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e000b490 0x7f96e000d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 -- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0008860 msgr2=0x7f96e000ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0008860 0x7f96e000ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f77fe640 1 -- 172.21.15.32:0/1746563245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0003450 con 0x7f96e0003a70 2023-10-16T08:17:31.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f6ffd640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0008860 0x7f96e000ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:31.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.265+0000 7f96f7fff640 1 -- 172.21.15.32:0/1746563245 reap_dead start 2023-10-16T08:17:31.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96f77fe640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e0003a70 0x7f96e0003e90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c240 tx=0x7f96e801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7011ec289108b868 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:31.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8005ae0 con 0x7f96e0003a70 2023-10-16T08:17:31.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8005c40 con 0x7f96e0003a70 2023-10-16T08:17:31.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96f67fc640 1 -- 172.21.15.32:0/1746563245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80072b0 con 0x7f96e0003a70 2023-10-16T08:17:31.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 -- 172.21.15.32:0/1746563245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e0003a70 msgr2=0x7f96e0003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e0003a70 0x7f96e0003e90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c240 tx=0x7f96e801cbc0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 -- 172.21.15.32:0/1746563245 shutdown_connections 2023-10-16T08:17:31.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 --2- 172.21.15.32:0/1746563245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e0003a70 0x7f96e0003e90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 -- 172.21.15.32:0/1746563245 >> 172.21.15.32:0/1746563245 conn(0x7f96f806c190 msgr2=0x7f96f806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:31.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 -- 172.21.15.32:0/1746563245 shutdown_connections 2023-10-16T08:17:31.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.269+0000 7f96fda91640 1 -- 172.21.15.32:0/1746563245 wait complete. 2023-10-16T08:17:31.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 Processor -- start 2023-10-16T08:17:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 -- start start 2023-10-16T08:17:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e0003a70 0x7f96f8151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81e5b10 0x7f96f8150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 0x7f96f81583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f81588f0 con 0x7f96f8157f30 2023-10-16T08:17:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80b4340 con 0x7f96f81e5b10 2023-10-16T08:17:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8151b70 con 0x7f96e0003a70 2023-10-16T08:17:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 0x7f96f81583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 0x7f96f81583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40798/0 (socket says 172.21.15.32:40798) 2023-10-16T08:17:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 -- 172.21.15.32:0/296670415 learned_addr learned my addr 172.21.15.32:0/296670415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f77fe640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e0003a70 0x7f96f8151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f6ffd640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81e5b10 0x7f96f8150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 -- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e0003a70 msgr2=0x7f96f8151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e0003a70 0x7f96f8151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 -- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81e5b10 msgr2=0x7f96f8150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81e5b10 0x7f96f8150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 -- 172.21.15.32:0/296670415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f81e60d0 con 0x7f96f8157f30 2023-10-16T08:17:31.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96f7fff640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 0x7f96f81583b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f96f80b0dc0 tx=0x7f96f00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:31.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000e070 con 0x7f96f8157f30 2023-10-16T08:17:31.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0011e10 con 0x7f96f8157f30 2023-10-16T08:17:31.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f80b3860 con 0x7f96f8157f30 2023-10-16T08:17:31.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f00039a0 con 0x7f96f8157f30 2023-10-16T08:17:31.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.273+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f80b3e20 con 0x7f96f8157f30 2023-10-16T08:17:31.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.277+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f8074010 con 0x7f96f8157f30 2023-10-16T08:17:31.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.277+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96f0003b00 con 0x7f96f8157f30 2023-10-16T08:17:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.277+0000 7f96dffff640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96e40664a0 0x7f96e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.277+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f96f0084700 con 0x7f96f8157f30 2023-10-16T08:17:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.277+0000 7f96f77fe640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96e40664a0 0x7f96e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.277+0000 7f96f77fe640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96e40664a0 0x7f96e4068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f96e80ab000 tx=0x7f96e80b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.281+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96f00517d0 con 0x7f96f8157f30 2023-10-16T08:17:31.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96f80cbc10 con 0x7f96f8157f30 2023-10-16T08:17:31.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96dffff640 1 -- 172.21.15.32:0/296670415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f96f0054df0 con 0x7f96f8157f30 2023-10-16T08:17:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96e40664a0 msgr2=0x7f96e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96e40664a0 0x7f96e4068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f96e80ab000 tx=0x7f96e80b1040 comp rx=0 tx=0).stop 2023-10-16T08:17:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 msgr2=0x7f96f81583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 0x7f96f81583b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f96f80b0dc0 tx=0x7f96f00115a0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 shutdown_connections 2023-10-16T08:17:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96f8157f30 0x7f96f81583b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81e5b10 0x7f96f8150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96e40664a0 0x7f96e4068960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 --2- 172.21.15.32:0/296670415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96e0003a70 0x7f96f8151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 >> 172.21.15.32:0/296670415 conn(0x7f96f806c190 msgr2=0x7f96f81539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 shutdown_connections 2023-10-16T08:17:31.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.449+0000 7f96fda91640 1 -- 172.21.15.32:0/296670415 wait complete. 2023-10-16T08:17:31.476 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:17:31.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:31.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-16T08:17:31.499 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:31.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:17:31.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:31.500 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T08:17:31.551 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:17:31.551 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:31.562 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:17:31.563 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:31.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 Processor -- start 2023-10-16T08:17:31.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 -- start start 2023-10-16T08:17:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf2c15ca50 0x7fcf2c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcf2c0b21c0 0x7fcf2c0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c1607e0 con 0x7fcf2c15ce30 2023-10-16T08:17:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c0cb4a0 con 0x7fcf2c15ca50 2023-10-16T08:17:31.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf342d6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c0cf060 con 0x7fcf2c0b21c0 2023-10-16T08:17:31.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf2c15ca50 0x7fcf2c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf2c15ca50 0x7fcf2c06c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40814/0 (socket says 172.21.15.32:40814) 2023-10-16T08:17:31.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 -- 172.21.15.32:0/3288665375 learned_addr learned my addr 172.21.15.32:0/3288665375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:31.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf31871640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf2c15ce30 0x7fcf2c0cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45832/0 (socket says 172.21.15.32:45832) 2023-10-16T08:17:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 -- 172.21.15.32:0/3288665375 >> v2:172.21.15.186:3300/0 conn(0x7fcf2c0b21c0 msgr2=0x7fcf2c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32873640 1 --2- 172.21.15.32:0/3288665375 >> v2:172.21.15.186:3300/0 conn(0x7fcf2c0b21c0 0x7fcf2c0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 --2- 172.21.15.32:0/3288665375 >> v2:172.21.15.186:3300/0 conn(0x7fcf2c0b21c0 0x7fcf2c0b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf31070640 1 -- 172.21.15.32:0/3288665375 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755295034 0 0) 0x7fcf2c1607e0 con 0x7fcf2c15ce30 2023-10-16T08:17:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 -- 172.21.15.32:0/3288665375 >> v1:172.21.15.32:6789/0 conn(0x7fcf2c15ce30 legacy=0x7fcf2c0cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32072640 1 -- 172.21.15.32:0/3288665375 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c0b2ae0 con 0x7fcf2c15ca50 2023-10-16T08:17:31.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.753+0000 7fcf32873640 1 --2- 172.21.15.32:0/3288665375 >> v2:172.21.15.186:3300/0 conn(0x7fcf2c0b21c0 0x7fcf2c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:31.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf32072640 1 --2- 172.21.15.32:0/3288665375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c06c980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0c4210 tx=0x7fcf2801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101432bffbcf3da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:31.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf31070640 1 -- 172.21.15.32:0/3288665375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280aa430 con 0x7fcf2c15ca50 2023-10-16T08:17:31.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf31070640 1 -- 172.21.15.32:0/3288665375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf280aa590 con 0x7fcf2c15ca50 2023-10-16T08:17:31.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf31070640 1 -- 172.21.15.32:0/3288665375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280aa920 con 0x7fcf2c15ca50 2023-10-16T08:17:31.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 -- 172.21.15.32:0/3288665375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 msgr2=0x7fcf2c06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 --2- 172.21.15.32:0/3288665375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c06c980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0c4210 tx=0x7fcf2801c4d0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 -- 172.21.15.32:0/3288665375 shutdown_connections 2023-10-16T08:17:31.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 --2- 172.21.15.32:0/3288665375 >> v2:172.21.15.186:3300/0 conn(0x7fcf2c0b21c0 0x7fcf2c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 --2- 172.21.15.32:0/3288665375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c06c980 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 -- 172.21.15.32:0/3288665375 >> 172.21.15.32:0/3288665375 conn(0x7fcf2c0cd1c0 msgr2=0x7fcf2c0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:31.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 -- 172.21.15.32:0/3288665375 shutdown_connections 2023-10-16T08:17:31.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.757+0000 7fcf342d6640 1 -- 172.21.15.32:0/3288665375 wait complete. 2023-10-16T08:17:31.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 Processor -- start 2023-10-16T08:17:31.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- start start 2023-10-16T08:17:31.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf2c0b21c0 0x7fcf2c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf2c15ce30 0x7fcf2c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15b680 con 0x7fcf2c15ca50 2023-10-16T08:17:31.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c159f10 con 0x7fcf2c15ce30 2023-10-16T08:17:31.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15b3a0 con 0x7fcf2c0b21c0 2023-10-16T08:17:31.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c14ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40824/0 (socket says 172.21.15.32:40824) 2023-10-16T08:17:31.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 -- 172.21.15.32:0/2103003941 learned_addr learned my addr 172.21.15.32:0/2103003941 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:31.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf32072640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf2c0b21c0 0x7fcf2c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf32873640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf2c15ce30 0x7fcf2c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 -- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf2c0b21c0 msgr2=0x7fcf2c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf2c0b21c0 0x7fcf2c15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 -- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf2c15ce30 msgr2=0x7fcf2c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf2c15ce30 0x7fcf2c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 -- 172.21.15.32:0/2103003941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280094d0 con 0x7fcf2c15ca50 2023-10-16T08:17:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf32072640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf2c0b21c0 0x7fcf2c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf31871640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c14ef50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c480 tx=0x7fcf1c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0074e0 con 0x7fcf2c15ca50 2023-10-16T08:17:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c007660 con 0x7fcf2c15ca50 2023-10-16T08:17:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c1598c0 con 0x7fcf2c15ca50 2023-10-16T08:17:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c004a30 con 0x7fcf2c15ca50 2023-10-16T08:17:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c151de0 con 0x7fcf2c15ca50 2023-10-16T08:17:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.761+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c159720 con 0x7fcf2c15ca50 2023-10-16T08:17:31.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.765+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf1c004b90 con 0x7fcf2c15ca50 2023-10-16T08:17:31.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.765+0000 7fcf22ffd640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf100664a0 0x7fcf10068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.765+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcf1c083780 con 0x7fcf2c15ca50 2023-10-16T08:17:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.765+0000 7fcf32072640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf100664a0 0x7fcf10068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.765+0000 7fcf32072640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf100664a0 0x7fcf10068960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ae000 tx=0x7fcf280a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:31.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.769+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf2c159720 con 0x7fcf2c15ca50 2023-10-16T08:17:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.925+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf2c159720 con 0x7fcf2c15ca50 2023-10-16T08:17:31.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.925+0000 7fcf22ffd640 1 -- 172.21.15.32:0/2103003941 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fcf2c159720 con 0x7fcf2c15ca50 2023-10-16T08:17:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:31.934 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:31.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf100664a0 msgr2=0x7fcf10068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf100664a0 0x7fcf10068960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ae000 tx=0x7fcf280a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:31.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 msgr2=0x7fcf2c14ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c14ef50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c480 tx=0x7fcf1c00c940 comp rx=0 tx=0).stop 2023-10-16T08:17:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 shutdown_connections 2023-10-16T08:17:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf2c15ce30 0x7fcf2c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf2c15ca50 0x7fcf2c14ef50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcf100664a0 0x7fcf10068960 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 --2- 172.21.15.32:0/2103003941 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcf2c0b21c0 0x7fcf2c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 >> 172.21.15.32:0/2103003941 conn(0x7fcf2c0cd1c0 msgr2=0x7fcf2c0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 shutdown_connections 2023-10-16T08:17:31.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:31.929+0000 7fcf342d6640 1 -- 172.21.15.32:0/2103003941 wait complete. 2023-10-16T08:17:31.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:31.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 Processor -- start 2023-10-16T08:17:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 -- start start 2023-10-16T08:17:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f08440d0210 0x7f08440d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08440b84e0 0x7f08440b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08440d1670 con 0x7f084415cba0 2023-10-16T08:17:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08440d1460 con 0x7f08440d0210 2023-10-16T08:17:32.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f084a2f7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08440d1950 con 0x7f08440b84e0 2023-10-16T08:17:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f08440d0210 0x7f08440d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f08440d0210 0x7f08440d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:41088/0 (socket says 172.21.15.32:41088) 2023-10-16T08:17:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 -- 172.21.15.32:0/243437324 learned_addr learned my addr 172.21.15.32:0/243437324 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f0848894640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08440b84e0 0x7f08440b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 -- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08440b84e0 msgr2=0x7f08440b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 --2- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08440b84e0 0x7f08440b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 -- 172.21.15.32:0/243437324 >> v1:172.21.15.186:6789/0 conn(0x7f084415cba0 legacy=0x7f08440cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.097+0000 7f08437fe640 1 -- 172.21.15.32:0/243437324 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440d1c30 con 0x7f08440d0210 2023-10-16T08:17:32.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f08437fe640 1 --2- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440d0210 0x7f08440d0f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0838009870 tx=0x7f083801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c3ed0d78f20ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f0842ffd640 1 -- 172.21.15.32:0/243437324 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380a9400 con 0x7f08440d0210 2023-10-16T08:17:32.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f0842ffd640 1 -- 172.21.15.32:0/243437324 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08380a9560 con 0x7f08440d0210 2023-10-16T08:17:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f0842ffd640 1 -- 172.21.15.32:0/243437324 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380a98f0 con 0x7f08440d0210 2023-10-16T08:17:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 -- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440d0210 msgr2=0x7f08440d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 --2- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440d0210 0x7f08440d0f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0838009870 tx=0x7f083801c540 comp rx=0 tx=0).stop 2023-10-16T08:17:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 -- 172.21.15.32:0/243437324 shutdown_connections 2023-10-16T08:17:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 --2- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f08440b84e0 0x7f08440b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 --2- 172.21.15.32:0/243437324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440d0210 0x7f08440d0f20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 -- 172.21.15.32:0/243437324 >> 172.21.15.32:0/243437324 conn(0x7f08440719c0 msgr2=0x7f0844071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 -- 172.21.15.32:0/243437324 shutdown_connections 2023-10-16T08:17:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 -- 172.21.15.32:0/243437324 wait complete. 2023-10-16T08:17:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.101+0000 7f084a2f7640 1 Processor -- start 2023-10-16T08:17:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- start start 2023-10-16T08:17:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440b84e0 0x7f0844156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 0x7f08441503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f084415cba0 0x7f084414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08441567a0 con 0x7f08440d0210 2023-10-16T08:17:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0844155b20 con 0x7f08440b84e0 2023-10-16T08:17:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0844155e00 con 0x7f084415cba0 2023-10-16T08:17:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 0x7f08441503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 0x7f08441503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40834/0 (socket says 172.21.15.32:40834) 2023-10-16T08:17:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 -- 172.21.15.32:0/3892201389 learned_addr learned my addr 172.21.15.32:0/3892201389 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f0848894640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f084415cba0 0x7f084414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f0843fff640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440b84e0 0x7f0844156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 -- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f084415cba0 msgr2=0x7f084414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f084415cba0 0x7f084414f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 -- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440b84e0 msgr2=0x7f0844156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440b84e0 0x7f0844156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 -- 172.21.15.32:0/3892201389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380094d0 con 0x7f08440d0210 2023-10-16T08:17:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f0843fff640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440b84e0 0x7f0844156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f08437fe640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 0x7f08441503b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f083801abf0 tx=0x7f083801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380a9400 con 0x7f08440d0210 2023-10-16T08:17:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08380b1820 con 0x7f08440d0210 2023-10-16T08:17:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380b1bd0 con 0x7f08440d0210 2023-10-16T08:17:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08441560e0 con 0x7f08440d0210 2023-10-16T08:17:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0844148480 con 0x7f08440d0210 2023-10-16T08:17:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.105+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08441568e0 con 0x7f08440d0210 2023-10-16T08:17:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.109+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08380b1d30 con 0x7f08440d0210 2023-10-16T08:17:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.109+0000 7f0840ff9640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f08300664a0 0x7f0830068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.109+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f083811eb10 con 0x7f08440d0210 2023-10-16T08:17:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.109+0000 7f0843fff640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f08300664a0 0x7f0830068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.109+0000 7f0843fff640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f08300664a0 0x7f0830068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f084414ada0 tx=0x7f0834009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.113+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f08441568e0 con 0x7f08440d0210 2023-10-16T08:17:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08441568e0 con 0x7f08440d0210 2023-10-16T08:17:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f0840ff9640 1 -- 172.21.15.32:0/3892201389 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08441568e0 con 0x7f08440d0210 2023-10-16T08:17:32.275 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f08300664a0 msgr2=0x7f0830068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f08300664a0 0x7f0830068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f084414ada0 tx=0x7f0834009040 comp rx=0 tx=0).stop 2023-10-16T08:17:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 msgr2=0x7f08441503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 0x7f08441503b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f083801abf0 tx=0x7f083801ad80 comp rx=0 tx=0).stop 2023-10-16T08:17:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 shutdown_connections 2023-10-16T08:17:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f08300664a0 0x7f0830068960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f084415cba0 0x7f084414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08440d0210 0x7f08441503b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 --2- 172.21.15.32:0/3892201389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f08440b84e0 0x7f0844156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 >> 172.21.15.32:0/3892201389 conn(0x7f08440719c0 msgr2=0x7f0844154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 shutdown_connections 2023-10-16T08:17:32.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.269+0000 7f084a2f7640 1 -- 172.21.15.32:0/3892201389 wait complete. 2023-10-16T08:17:32.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 Processor -- start 2023-10-16T08:17:32.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 -- start start 2023-10-16T08:17:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb48015ce10 0x7fb480161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb48006e2f0 0x7fb48006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4800cdc30 con 0x7fb4800cd3e0 2023-10-16T08:17:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48006ecc0 con 0x7fb48006e2f0 2023-10-16T08:17:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb484d3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800caec0 con 0x7fb48015ce10 2023-10-16T08:17:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb48006e2f0 0x7fb48006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb48006e2f0 0x7fb48006e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40836/0 (socket says 172.21.15.32:40836) 2023-10-16T08:17:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 -- 172.21.15.32:0/2533623589 learned_addr learned my addr 172.21.15.32:0/2533623589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47e59c640 1 --2- 172.21.15.32:0/2533623589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb48015ce10 0x7fb480161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 -- 172.21.15.32:0/2533623589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb48015ce10 msgr2=0x7fb480161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 --2- 172.21.15.32:0/2533623589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb48015ce10 0x7fb480161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 -- 172.21.15.32:0/2533623589 >> v1:172.21.15.186:6789/0 conn(0x7fb4800cd3e0 legacy=0x7fb4800ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.437+0000 7fb47ed9d640 1 -- 172.21.15.32:0/2533623589 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800c9750 con 0x7fb48006e2f0 2023-10-16T08:17:32.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb47ed9d640 1 --2- 172.21.15.32:0/2533623589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb48006e780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb478009e50 tx=0x7fb47801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e85664055e45a2b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb47d59a640 1 -- 172.21.15.32:0/2533623589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780a58e0 con 0x7fb48006e2f0 2023-10-16T08:17:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb47d59a640 1 -- 172.21.15.32:0/2533623589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4780a5a40 con 0x7fb48006e2f0 2023-10-16T08:17:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb47d59a640 1 -- 172.21.15.32:0/2533623589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780a5dd0 con 0x7fb48006e2f0 2023-10-16T08:17:32.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 -- 172.21.15.32:0/2533623589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 msgr2=0x7fb48006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 --2- 172.21.15.32:0/2533623589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb48006e780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb478009e50 tx=0x7fb47801ca10 comp rx=0 tx=0).stop 2023-10-16T08:17:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 -- 172.21.15.32:0/2533623589 shutdown_connections 2023-10-16T08:17:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 --2- 172.21.15.32:0/2533623589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb48006e780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 --2- 172.21.15.32:0/2533623589 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb48015ce10 0x7fb480161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 -- 172.21.15.32:0/2533623589 >> 172.21.15.32:0/2533623589 conn(0x7fb4800cef20 msgr2=0x7fb4800cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 -- 172.21.15.32:0/2533623589 shutdown_connections 2023-10-16T08:17:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.441+0000 7fb484d3e640 1 -- 172.21.15.32:0/2533623589 wait complete. 2023-10-16T08:17:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 Processor -- start 2023-10-16T08:17:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 -- start start 2023-10-16T08:17:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb480147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4800cd3e0 0x7fb4800c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4800c55d0 0x7fb48015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800c5a60 con 0x7fb48006e2f0 2023-10-16T08:17:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48015e890 con 0x7fb4800c55d0 2023-10-16T08:17:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48015eb70 con 0x7fb4800cd3e0 2023-10-16T08:17:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb480147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb480147200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40844/0 (socket says 172.21.15.32:40844) 2023-10-16T08:17:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 -- 172.21.15.32:0/424334803 learned_addr learned my addr 172.21.15.32:0/424334803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47ed9d640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4800c55d0 0x7fb48015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47dd9b640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4800cd3e0 0x7fb4800c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 -- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4800cd3e0 msgr2=0x7fb4800c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4800cd3e0 0x7fb4800c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 -- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4800c55d0 msgr2=0x7fb48015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4800c55d0 0x7fb48015dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 -- 172.21.15.32:0/424334803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478009b30 con 0x7fb48006e2f0 2023-10-16T08:17:32.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb47e59c640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb480147200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb47000e870 tx=0x7fb47000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470009510 con 0x7fb48006e2f0 2023-10-16T08:17:32.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47000eec0 con 0x7fb48006e2f0 2023-10-16T08:17:32.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000cb40 con 0x7fb48006e2f0 2023-10-16T08:17:32.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4800c29e0 con 0x7fb48006e2f0 2023-10-16T08:17:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.445+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4800c2fa0 con 0x7fb48006e2f0 2023-10-16T08:17:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.449+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48004e170 con 0x7fb48006e2f0 2023-10-16T08:17:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.449+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb47000ccc0 con 0x7fb48006e2f0 2023-10-16T08:17:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.449+0000 7fb46effd640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4600664a0 0x7fb460068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.449+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb470084310 con 0x7fb48006e2f0 2023-10-16T08:17:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.449+0000 7fb47dd9b640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4600664a0 0x7fb460068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.449+0000 7fb47dd9b640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4600664a0 0x7fb460068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4800714f0 tx=0x7fb474009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.453+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb4700513e0 con 0x7fb48006e2f0 2023-10-16T08:17:32.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.609+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb48006ef60 con 0x7fb48006e2f0 2023-10-16T08:17:32.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.609+0000 7fb46effd640 1 -- 172.21.15.32:0/424334803 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb470054a00 con 0x7fb48006e2f0 2023-10-16T08:17:32.617 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4600664a0 msgr2=0x7fb460068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4600664a0 0x7fb460068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4800714f0 tx=0x7fb474009040 comp rx=0 tx=0).stop 2023-10-16T08:17:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 msgr2=0x7fb480147200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb480147200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb47000e870 tx=0x7fb47000ec70 comp rx=0 tx=0).stop 2023-10-16T08:17:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 shutdown_connections 2023-10-16T08:17:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4800c55d0 0x7fb48015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb4600664a0 0x7fb460068960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb4800cd3e0 0x7fb4800c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 --2- 172.21.15.32:0/424334803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb48006e2f0 0x7fb480147200 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 >> 172.21.15.32:0/424334803 conn(0x7fb4800cef20 msgr2=0x7fb4800cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 shutdown_connections 2023-10-16T08:17:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.613+0000 7fb484d3e640 1 -- 172.21.15.32:0/424334803 wait complete. 2023-10-16T08:17:32.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7041405640 1 Processor -- start 2023-10-16T08:17:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7041405640 1 -- start start 2023-10-16T08:17:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7041405640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f703c15cba0 0x7f703c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7041405640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c0d1670 con 0x7f703c0d0210 2023-10-16T08:17:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7041405640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c0d1950 con 0x7f703c0b84e0 2023-10-16T08:17:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7041405640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703c0d1460 con 0x7f703c15cba0 2023-10-16T08:17:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f703b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f703c0d0210 0x7f703c0b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45844/0 (socket says 172.21.15.32:45844) 2023-10-16T08:17:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f703b7fe640 1 -- 172.21.15.32:0/3924552085 learned_addr learned my addr 172.21.15.32:0/3924552085 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f703affd640 1 --2- 172.21.15.32:0/3924552085 >> v2:172.21.15.186:3300/0 conn(0x7f703c15cba0 0x7f703c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016257417 0 0) 0x7f703c0d1670 con 0x7f703c0d0210 2023-10-16T08:17:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7024003530 con 0x7f703c0d0210 2023-10-16T08:17:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273071663 0 0) 0x7f703c0d1950 con 0x7f703c0b84e0 2023-10-16T08:17:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f703c0d1670 con 0x7f703c0b84e0 2023-10-16T08:17:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f703affd640 1 -- 172.21.15.32:0/3924552085 >> v1:172.21.15.186:6789/0 conn(0x7f703c0b84e0 legacy=0x7f703c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f703affd640 1 -- 172.21.15.32:0/3924552085 >> v1:172.21.15.32:6789/0 conn(0x7f703c0d0210 legacy=0x7f703c0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.781+0000 7f703affd640 1 -- 172.21.15.32:0/3924552085 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0d1c30 con 0x7f703c15cba0 2023-10-16T08:17:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f703affd640 1 --2- 172.21.15.32:0/3924552085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c15cba0 0x7f703c0d0f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f702c0097c0 tx=0x7f702c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab624be85744a7b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0ab5f0 con 0x7f703c15cba0 2023-10-16T08:17:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c0ab750 con 0x7f703c15cba0 2023-10-16T08:17:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7039ffb640 1 -- 172.21.15.32:0/3924552085 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0abae0 con 0x7f703c15cba0 2023-10-16T08:17:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 -- 172.21.15.32:0/3924552085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c15cba0 msgr2=0x7f703c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 --2- 172.21.15.32:0/3924552085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c15cba0 0x7f703c0d0f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f702c0097c0 tx=0x7f702c01c3f0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 -- 172.21.15.32:0/3924552085 shutdown_connections 2023-10-16T08:17:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 --2- 172.21.15.32:0/3924552085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c15cba0 0x7f703c0d0f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 -- 172.21.15.32:0/3924552085 >> 172.21.15.32:0/3924552085 conn(0x7f703c0719c0 msgr2=0x7f703c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 -- 172.21.15.32:0/3924552085 shutdown_connections 2023-10-16T08:17:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.785+0000 7f7041405640 1 -- 172.21.15.32:0/3924552085 wait complete. 2023-10-16T08:17:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 Processor -- start 2023-10-16T08:17:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 -- start start 2023-10-16T08:17:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 0x7f703c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c0d0210 0x7f703c15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f703c15cba0 0x7f703c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0c65e0 con 0x7f703c0b84e0 2023-10-16T08:17:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0c6880 con 0x7f703c0d0210 2023-10-16T08:17:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0c6b60 con 0x7f703c15cba0 2023-10-16T08:17:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 0x7f703c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 0x7f703c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40858/0 (socket says 172.21.15.32:40858) 2023-10-16T08:17:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 -- 172.21.15.32:0/839233315 learned_addr learned my addr 172.21.15.32:0/839233315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 -- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f703c15cba0 msgr2=0x7f703c14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f703c15cba0 0x7f703c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 -- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c0d0210 msgr2=0x7f703c15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c0d0210 0x7f703c15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 -- 172.21.15.32:0/839233315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c009420 con 0x7f703c0b84e0 2023-10-16T08:17:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f703affd640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 0x7f703c0c8d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f702c0ad040 tx=0x7f702c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0bf610 con 0x7f703c0b84e0 2023-10-16T08:17:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c0bf7b0 con 0x7f703c0b84e0 2023-10-16T08:17:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0bfb60 con 0x7f703c0b84e0 2023-10-16T08:17:32.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c0c6e40 con 0x7f703c0b84e0 2023-10-16T08:17:32.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.789+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c14c500 con 0x7f703c0b84e0 2023-10-16T08:17:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.793+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c0c7080 con 0x7f703c0b84e0 2023-10-16T08:17:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.793+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f702c0b5400 con 0x7f703c0b84e0 2023-10-16T08:17:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.793+0000 7f70237fe640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f70280664a0 0x7f7028068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.793+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f702c0a8050 con 0x7f703c0b84e0 2023-10-16T08:17:32.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.793+0000 7f703a7fc640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f70280664a0 0x7f7028068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.793+0000 7f703a7fc640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f70280664a0 0x7f7028068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7030000f70 tx=0x7f7030007920 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.797+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f703c0c7080 con 0x7f703c0b84e0 2023-10-16T08:17:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f703c0c7080 con 0x7f703c0b84e0 2023-10-16T08:17:32.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f70237fe640 1 -- 172.21.15.32:0/839233315 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f703c0c7080 con 0x7f703c0b84e0 2023-10-16T08:17:32.960 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f70280664a0 msgr2=0x7f7028068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f70280664a0 0x7f7028068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7030000f70 tx=0x7f7030007920 comp rx=0 tx=0).stop 2023-10-16T08:17:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 msgr2=0x7f703c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 0x7f703c0c8d90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f702c0ad040 tx=0x7f702c01ac30 comp rx=0 tx=0).stop 2023-10-16T08:17:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 shutdown_connections 2023-10-16T08:17:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f70280664a0 0x7f7028068960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f703c15cba0 0x7f703c14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f703c0d0210 0x7f703c15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 --2- 172.21.15.32:0/839233315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f703c0b84e0 0x7f703c0c8d90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 >> 172.21.15.32:0/839233315 conn(0x7f703c0719c0 msgr2=0x7f703c15b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 shutdown_connections 2023-10-16T08:17:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:32.953+0000 7f7041405640 1 -- 172.21.15.32:0/839233315 wait complete. 2023-10-16T08:17:32.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 Processor -- start 2023-10-16T08:17:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 -- start start 2023-10-16T08:17:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efe7006d010 0x7efe700c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe700c26b0 0x7efe700c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7015ae30 con 0x7efe70069e90 2023-10-16T08:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700cb090 con 0x7efe7006d010 2023-10-16T08:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe767f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7006d470 con 0x7efe700c26b0 2023-10-16T08:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe6ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe70069e90 0x7efe7006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42152/0 (socket says 172.21.15.32:42152) 2023-10-16T08:17:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe6ffff640 1 -- 172.21.15.32:0/4077603932 learned_addr learned my addr 172.21.15.32:0/4077603932 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe74d95640 1 --2- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe700c26b0 0x7efe700c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.121+0000 7efe6f7fe640 1 --2- 172.21.15.32:0/4077603932 >> v2:172.21.15.186:3300/0 conn(0x7efe7006d010 0x7efe700c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6effd640 1 -- 172.21.15.32:0/4077603932 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942273149 0 0) 0x7efe7015ae30 con 0x7efe70069e90 2023-10-16T08:17:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6effd640 1 -- 172.21.15.32:0/4077603932 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe58003530 con 0x7efe70069e90 2023-10-16T08:17:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6f7fe640 1 -- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe700c26b0 msgr2=0x7efe700c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6effd640 1 -- 172.21.15.32:0/4077603932 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737989692 0 0) 0x7efe58003530 con 0x7efe70069e90 2023-10-16T08:17:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6f7fe640 1 --2- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe700c26b0 0x7efe700c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6f7fe640 1 -- 172.21.15.32:0/4077603932 >> v1:172.21.15.32:6789/0 conn(0x7efe70069e90 legacy=0x7efe7006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6f7fe640 1 -- 172.21.15.32:0/4077603932 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700ca510 con 0x7efe7006d010 2023-10-16T08:17:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6f7fe640 1 --2- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c9ce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efe640097c0 tx=0x7efe6401eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feca887a4a65fa66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6effd640 1 -- 172.21.15.32:0/4077603932 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60001680 con 0x7efe7006d010 2023-10-16T08:17:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6effd640 1 -- 172.21.15.32:0/4077603932 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe640a85d0 con 0x7efe7006d010 2023-10-16T08:17:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.125+0000 7efe6effd640 1 -- 172.21.15.32:0/4077603932 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640a8960 con 0x7efe7006d010 2023-10-16T08:17:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 msgr2=0x7efe700c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 --2- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c9ce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efe640097c0 tx=0x7efe6401eb90 comp rx=0 tx=0).stop 2023-10-16T08:17:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- 172.21.15.32:0/4077603932 shutdown_connections 2023-10-16T08:17:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 --2- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe700c26b0 0x7efe700c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 --2- 172.21.15.32:0/4077603932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c9ce0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- 172.21.15.32:0/4077603932 >> 172.21.15.32:0/4077603932 conn(0x7efe700b0da0 msgr2=0x7efe700ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:33.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- 172.21.15.32:0/4077603932 shutdown_connections 2023-10-16T08:17:33.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- 172.21.15.32:0/4077603932 wait complete. 2023-10-16T08:17:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 Processor -- start 2023-10-16T08:17:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- start start 2023-10-16T08:17:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe70069e90 0x7efe7014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 0x7efe7015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7014ed00 con 0x7efe700c26b0 2023-10-16T08:17:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe701efac0 con 0x7efe7006d010 2023-10-16T08:17:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe767f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700c6300 con 0x7efe70069e90 2023-10-16T08:17:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe74d95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 0x7efe7015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe74d95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 0x7efe7015cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59270/0 (socket says 172.21.15.32:59270) 2023-10-16T08:17:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.129+0000 7efe74d95640 1 -- 172.21.15.32:0/3303106458 learned_addr learned my addr 172.21.15.32:0/3303106458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6f7fe640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6ffff640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe70069e90 0x7efe7014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe74d95640 1 -- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe70069e90 msgr2=0x7efe7014e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe74d95640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe70069e90 0x7efe7014e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe74d95640 1 -- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 msgr2=0x7efe700c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe74d95640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe74d95640 1 -- 172.21.15.32:0/3303106458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe64009420 con 0x7efe700c26b0 2023-10-16T08:17:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6f7fe640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe74d95640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 0x7efe7015cdb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe6800efc0 tx=0x7efe6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68014480 con 0x7efe700c26b0 2023-10-16T08:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6800f040 con 0x7efe700c26b0 2023-10-16T08:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68014820 con 0x7efe700c26b0 2023-10-16T08:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe700c65a0 con 0x7efe700c26b0 2023-10-16T08:17:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe700c6b60 con 0x7efe700c26b0 2023-10-16T08:17:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7004e170 con 0x7efe700c26b0 2023-10-16T08:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efe68014980 con 0x7efe700c26b0 2023-10-16T08:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6cff9640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe5c0664a0 0x7efe5c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.133+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efe68084ec0 con 0x7efe700c26b0 2023-10-16T08:17:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.137+0000 7efe6ffff640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe5c0664a0 0x7efe5c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.137+0000 7efe6ffff640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe5c0664a0 0x7efe5c068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efe60001b00 tx=0x7efe60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.137+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efe68051f70 con 0x7efe700c26b0 2023-10-16T08:17:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.293+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efe700c2090 con 0x7efe700c26b0 2023-10-16T08:17:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe6cff9640 1 -- 172.21.15.32:0/3303106458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7efe68018020 con 0x7efe700c26b0 2023-10-16T08:17:33.303 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe5c0664a0 msgr2=0x7efe5c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe5c0664a0 0x7efe5c068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efe60001b00 tx=0x7efe60009040 comp rx=0 tx=0).stop 2023-10-16T08:17:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 msgr2=0x7efe7015cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 0x7efe7015cdb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe6800efc0 tx=0x7efe6800c490 comp rx=0 tx=0).stop 2023-10-16T08:17:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 shutdown_connections 2023-10-16T08:17:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe700c26b0 0x7efe7015cdb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe7006d010 0x7efe700c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe5c0664a0 0x7efe5c068960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 --2- 172.21.15.32:0/3303106458 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe70069e90 0x7efe7014e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 >> 172.21.15.32:0/3303106458 conn(0x7efe700b0da0 msgr2=0x7efe7006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 shutdown_connections 2023-10-16T08:17:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.297+0000 7efe767f8640 1 -- 172.21.15.32:0/3303106458 wait complete. 2023-10-16T08:17:33.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.461+0000 7f96a55a1640 1 Processor -- start 2023-10-16T08:17:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.461+0000 7f96a55a1640 1 -- start start 2023-10-16T08:17:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f96a55a1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96a00cf090 0x7f96a00cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f96a55a1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f96a00c6ad0 0x7f96a00c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f96a55a1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00c65f0 con 0x7f96a015ca70 2023-10-16T08:17:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f96a55a1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00cfa60 con 0x7f96a00cf090 2023-10-16T08:17:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f96a55a1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00ce820 con 0x7f96a00c6ad0 2023-10-16T08:17:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96a00cf090 0x7f96a00cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96a00cf090 0x7f96a00cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59286/0 (socket says 172.21.15.32:59286) 2023-10-16T08:17:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 -- 172.21.15.32:0/56481651 learned_addr learned my addr 172.21.15.32:0/56481651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 -- 172.21.15.32:0/56481651 >> v2:172.21.15.186:3300/0 conn(0x7f96a00c6ad0 msgr2=0x7f96a00c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 --2- 172.21.15.32:0/56481651 >> v2:172.21.15.186:3300/0 conn(0x7f96a00c6ad0 0x7f96a00c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 -- 172.21.15.32:0/56481651 >> v1:172.21.15.186:6789/0 conn(0x7f96a015ca70 legacy=0x7f96a00d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 -- 172.21.15.32:0/56481651 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a00cdf80 con 0x7f96a00cf090 2023-10-16T08:17:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.465+0000 7f9697fff640 1 --2- 172.21.15.32:0/56481651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a00cf090 0x7f96a00cf520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b8950 tx=0x7f968801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2950b0a43bf0971 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969e7fc640 1 -- 172.21.15.32:0/56481651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880aa5f0 con 0x7f96a00cf090 2023-10-16T08:17:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969e7fc640 1 -- 172.21.15.32:0/56481651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96880aa750 con 0x7f96a00cf090 2023-10-16T08:17:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969e7fc640 1 -- 172.21.15.32:0/56481651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880aaae0 con 0x7f96a00cf090 2023-10-16T08:17:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- 172.21.15.32:0/56481651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a00cf090 msgr2=0x7f96a00cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 --2- 172.21.15.32:0/56481651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a00cf090 0x7f96a00cf520 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b8950 tx=0x7f968801c490 comp rx=0 tx=0).stop 2023-10-16T08:17:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- 172.21.15.32:0/56481651 shutdown_connections 2023-10-16T08:17:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 --2- 172.21.15.32:0/56481651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a00cf090 0x7f96a00cf520 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 --2- 172.21.15.32:0/56481651 >> v2:172.21.15.186:3300/0 conn(0x7f96a00c6ad0 0x7f96a00c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- 172.21.15.32:0/56481651 >> 172.21.15.32:0/56481651 conn(0x7f96a009bc50 msgr2=0x7f96a00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- 172.21.15.32:0/56481651 shutdown_connections 2023-10-16T08:17:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- 172.21.15.32:0/56481651 wait complete. 2023-10-16T08:17:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 Processor -- start 2023-10-16T08:17:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- start start 2023-10-16T08:17:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96a00c6ad0 0x7f96a00b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a00cf090 0x7f96a014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 0x7f96a00caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00b7cc0 con 0x7f96a015ca70 2023-10-16T08:17:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a014d8b0 con 0x7f96a00cf090 2023-10-16T08:17:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f96a55a1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00b54f0 con 0x7f96a00c6ad0 2023-10-16T08:17:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 0x7f96a00caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 0x7f96a00caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59288/0 (socket says 172.21.15.32:59288) 2023-10-16T08:17:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969f7fe640 1 -- 172.21.15.32:0/3670064510 learned_addr learned my addr 172.21.15.32:0/3670064510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f9697fff640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a00cf090 0x7f96a014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.469+0000 7f969effd640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96a00c6ad0 0x7f96a00b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f969f7fe640 1 -- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96a00c6ad0 msgr2=0x7f96a00b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f969f7fe640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96a00c6ad0 0x7f96a00b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f969f7fe640 1 -- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a00cf090 msgr2=0x7f96a014da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f969f7fe640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a00cf090 0x7f96a014da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f969f7fe640 1 -- 172.21.15.32:0/3670064510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880094d0 con 0x7f96a015ca70 2023-10-16T08:17:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f969f7fe640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 0x7f96a00caa10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f969800b9a0 tx=0x7f969800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800cba0 con 0x7f96a015ca70 2023-10-16T08:17:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969800cd20 con 0x7f96a015ca70 2023-10-16T08:17:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698014680 con 0x7f96a015ca70 2023-10-16T08:17:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a00d38a0 con 0x7f96a015ca70 2023-10-16T08:17:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a014e220 con 0x7f96a015ca70 2023-10-16T08:17:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a00b5630 con 0x7f96a015ca70 2023-10-16T08:17:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f969801d3b0 con 0x7f96a015ca70 2023-10-16T08:17:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96977fe640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96800664a0 0x7f9680068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.473+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9698084190 con 0x7f96a015ca70 2023-10-16T08:17:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.477+0000 7f969effd640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96800664a0 0x7f9680068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.477+0000 7f969effd640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96800664a0 0x7f9680068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96a0153250 tx=0x7f9690009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.477+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9698051260 con 0x7f96a015ca70 2023-10-16T08:17:33.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.645+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96a0074010 con 0x7f96a015ca70 2023-10-16T08:17:33.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.645+0000 7f96977fe640 1 -- 172.21.15.32:0/3670064510 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9698054880 con 0x7f96a015ca70 2023-10-16T08:17:33.653 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:33.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96800664a0 msgr2=0x7f9680068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96800664a0 0x7f9680068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96a0153250 tx=0x7f9690009060 comp rx=0 tx=0).stop 2023-10-16T08:17:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 msgr2=0x7f96a00caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 0x7f96a00caa10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f969800b9a0 tx=0x7f969800be60 comp rx=0 tx=0).stop 2023-10-16T08:17:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 shutdown_connections 2023-10-16T08:17:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96a015ca70 0x7f96a00caa10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a00cf090 0x7f96a014da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96800664a0 0x7f9680068960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 --2- 172.21.15.32:0/3670064510 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96a00c6ad0 0x7f96a00b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 >> 172.21.15.32:0/3670064510 conn(0x7f96a009bc50 msgr2=0x7f96a00b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 shutdown_connections 2023-10-16T08:17:33.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.649+0000 7f96a55a1640 1 -- 172.21.15.32:0/3670064510 wait complete. 2023-10-16T08:17:33.674 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:17:33.674 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:17:33.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.821+0000 7fc2c6ec9640 1 Processor -- start 2023-10-16T08:17:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.821+0000 7fc2c6ec9640 1 -- start start 2023-10-16T08:17:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c6ec9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 0x7fc2c015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c6ec9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc2c00b07f0 0x7fc2c0164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c6ec9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c015be60 con 0x7fc2c00b4020 2023-10-16T08:17:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c6ec9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c015b3b0 con 0x7fc2c00b07f0 2023-10-16T08:17:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c6ec9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c015c1f0 con 0x7fc2c0070c40 2023-10-16T08:17:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2c00b4020 0x7fc2c00b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42164/0 (socket says 172.21.15.32:42164) 2023-10-16T08:17:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bffff640 1 -- 172.21.15.32:0/4247062928 learned_addr learned my addr 172.21.15.32:0/4247062928 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 0x7fc2c015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c5466640 1 --2- 172.21.15.32:0/4247062928 >> v2:172.21.15.186:3300/0 conn(0x7fc2c00b07f0 0x7fc2c0164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941288538 0 0) 0x7fc2c015be60 con 0x7fc2c00b4020 2023-10-16T08:17:33.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b0003530 con 0x7fc2c00b4020 2023-10-16T08:17:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669215619 0 0) 0x7fc2b0003530 con 0x7fc2c00b4020 2023-10-16T08:17:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 msgr2=0x7fc2c015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 0x7fc2c015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 >> v2:172.21.15.186:3300/0 conn(0x7fc2c00b07f0 msgr2=0x7fc2c0164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 --2- 172.21.15.32:0/4247062928 >> v2:172.21.15.186:3300/0 conn(0x7fc2c00b07f0 0x7fc2c0164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0071020 con 0x7fc2c00b4020 2023-10-16T08:17:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fc2ac001680 con 0x7fc2c00b4020 2023-10-16T08:17:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 >> v1:172.21.15.32:6789/0 conn(0x7fc2c00b4020 legacy=0x7fc2c00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b0003980 0x7fc2b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b00047d0 0x7fc2b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 0x7fc2b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c015b3b0 con 0x7fc2b0003980 2023-10-16T08:17:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c015c1f0 con 0x7fc2b00047d0 2023-10-16T08:17:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000b990 con 0x7fc2c0070c40 2023-10-16T08:17:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b0003980 0x7fc2b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2bffff640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b00047d0 0x7fc2b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c5466640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 0x7fc2b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 -- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 msgr2=0x7fc2b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c0070c40 0x7fc2b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 -- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b00047d0 msgr2=0x7fc2b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b00047d0 0x7fc2b0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.825+0000 7fc2c4c65640 1 -- 172.21.15.32:0/4247062928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0003530 con 0x7fc2b0003980 2023-10-16T08:17:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b0003980 0x7fc2b0003da0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009b10 tx=0x7fc2b401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cc03d6caf3eb4f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40a5540 con 0x7fc2b0003980 2023-10-16T08:17:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b40a56a0 con 0x7fc2b0003980 2023-10-16T08:17:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2bf7fe640 1 -- 172.21.15.32:0/4247062928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40a5a50 con 0x7fc2b0003980 2023-10-16T08:17:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/4247062928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b0003980 msgr2=0x7fc2b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/4247062928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b0003980 0x7fc2b0003da0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009b10 tx=0x7fc2b401ca50 comp rx=0 tx=0).stop 2023-10-16T08:17:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c5466640 1 -- 172.21.15.32:0/4247062928 reap_dead start 2023-10-16T08:17:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/4247062928 shutdown_connections 2023-10-16T08:17:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/4247062928 >> 172.21.15.32:0/4247062928 conn(0x7fc2c006c190 msgr2=0x7fc2c006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/4247062928 shutdown_connections 2023-10-16T08:17:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/4247062928 wait complete. 2023-10-16T08:17:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 Processor -- start 2023-10-16T08:17:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.829+0000 7fc2c6ec9640 1 -- start start 2023-10-16T08:17:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b0003980 0x7fc2c01641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c00c3aa0 0x7fc2c0161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 0x7fc2c0150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c0161120 con 0x7fc2c014fda0 2023-10-16T08:17:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c0160510 con 0x7fc2b0003980 2023-10-16T08:17:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c01607f0 con 0x7fc2c00c3aa0 2023-10-16T08:17:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 0x7fc2c0150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 0x7fc2c0150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59300/0 (socket says 172.21.15.32:59300) 2023-10-16T08:17:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 -- 172.21.15.32:0/1680111030 learned_addr learned my addr 172.21.15.32:0/1680111030 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b0003980 0x7fc2c01641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2bffff640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c00c3aa0 0x7fc2c0161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 -- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c00c3aa0 msgr2=0x7fc2c0161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c00c3aa0 0x7fc2c0161940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 -- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b0003980 msgr2=0x7fc2c01641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b0003980 0x7fc2c01641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 -- 172.21.15.32:0/1680111030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0160ad0 con 0x7fc2c014fda0 2023-10-16T08:17:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c5466640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 0x7fc2c0150220 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0070120 tx=0x7fc2b8011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800e040 con 0x7fc2c014fda0 2023-10-16T08:17:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b800f040 con 0x7fc2c014fda0 2023-10-16T08:17:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c0160db0 con 0x7fc2c014fda0 2023-10-16T08:17:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800f7e0 con 0x7fc2c014fda0 2023-10-16T08:17:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c0161e80 con 0x7fc2c014fda0 2023-10-16T08:17:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.833+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c0160f40 con 0x7fc2c014fda0 2023-10-16T08:17:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.837+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc2b800f960 con 0x7fc2c014fda0 2023-10-16T08:17:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.837+0000 7fc2bd7fa640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc2a40664d0 0x7fc2a4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.837+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc2b8084860 con 0x7fc2c014fda0 2023-10-16T08:17:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.837+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc2a40664d0 0x7fc2a4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.837+0000 7fc2c4c65640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc2a40664d0 0x7fc2a4068990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40af000 tx=0x7fc2b40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:33.841+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc2b8055450 con 0x7fc2c014fda0 2023-10-16T08:17:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c0074010 con 0x7fc2c014fda0 2023-10-16T08:17:34.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2bd7fa640 1 -- 172.21.15.32:0/1680111030 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fc2b8054fe0 con 0x7fc2c014fda0 2023-10-16T08:17:34.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:34.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:34.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc2a40664d0 msgr2=0x7fc2a4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc2a40664d0 0x7fc2a4068990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40af000 tx=0x7fc2b40046d0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 msgr2=0x7fc2c0150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 0x7fc2c0150220 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0070120 tx=0x7fc2b8011f10 comp rx=0 tx=0).stop 2023-10-16T08:17:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 shutdown_connections 2023-10-16T08:17:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c014fda0 0x7fc2c0150220 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc2a40664d0 0x7fc2a4068990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc2c00c3aa0 0x7fc2c0161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 --2- 172.21.15.32:0/1680111030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2b0003980 0x7fc2c01641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 >> 172.21.15.32:0/1680111030 conn(0x7fc2c006c190 msgr2=0x7fc2c00afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:34.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 shutdown_connections 2023-10-16T08:17:34.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.005+0000 7fc2c6ec9640 1 -- 172.21.15.32:0/1680111030 wait complete. 2023-10-16T08:17:34.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:34.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 Processor -- start 2023-10-16T08:17:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 -- start start 2023-10-16T08:17:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f15a8069e90 0x7f15a806d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15a80c26b0 0x7f15a80c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a815ae30 con 0x7f15a806a270 2023-10-16T08:17:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80cb090 con 0x7f15a80c26b0 2023-10-16T08:17:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15afdf8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80ca510 con 0x7f15a8069e90 2023-10-16T08:17:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15a80c26b0 0x7f15a80c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15a80c26b0 0x7f15a80c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59314/0 (socket says 172.21.15.32:59314) 2023-10-16T08:17:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 -- 172.21.15.32:0/2452280073 learned_addr learned my addr 172.21.15.32:0/2452280073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15adb94640 1 --2- 172.21.15.32:0/2452280073 >> v2:172.21.15.186:3300/0 conn(0x7f15a8069e90 0x7f15a806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 -- 172.21.15.32:0/2452280073 >> v2:172.21.15.186:3300/0 conn(0x7f15a8069e90 msgr2=0x7f15a806d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 --2- 172.21.15.32:0/2452280073 >> v2:172.21.15.186:3300/0 conn(0x7f15a8069e90 0x7f15a806d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 -- 172.21.15.32:0/2452280073 >> v1:172.21.15.32:6789/0 conn(0x7f15a806a270 legacy=0x7f15a80c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.173+0000 7f15ae395640 1 -- 172.21.15.32:0/2452280073 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80ca7f0 con 0x7f15a80c26b0 2023-10-16T08:17:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15ae395640 1 --2- 172.21.15.32:0/2452280073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a80c2b00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f15a8071590 tx=0x7f15a001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b57a0239e47c8a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15acb92640 1 -- 172.21.15.32:0/2452280073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00a5860 con 0x7f15a80c26b0 2023-10-16T08:17:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15acb92640 1 -- 172.21.15.32:0/2452280073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a00a59c0 con 0x7f15a80c26b0 2023-10-16T08:17:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15acb92640 1 -- 172.21.15.32:0/2452280073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00a5d50 con 0x7f15a80c26b0 2023-10-16T08:17:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 -- 172.21.15.32:0/2452280073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 msgr2=0x7f15a80c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 --2- 172.21.15.32:0/2452280073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a80c2b00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f15a8071590 tx=0x7f15a001c9a0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 -- 172.21.15.32:0/2452280073 shutdown_connections 2023-10-16T08:17:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 --2- 172.21.15.32:0/2452280073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a80c2b00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 --2- 172.21.15.32:0/2452280073 >> v2:172.21.15.186:3300/0 conn(0x7f15a8069e90 0x7f15a806d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 -- 172.21.15.32:0/2452280073 >> 172.21.15.32:0/2452280073 conn(0x7f15a80b0da0 msgr2=0x7f15a80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 -- 172.21.15.32:0/2452280073 shutdown_connections 2023-10-16T08:17:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.177+0000 7f15afdf8640 1 -- 172.21.15.32:0/2452280073 wait complete. 2023-10-16T08:17:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 Processor -- start 2023-10-16T08:17:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 -- start start 2023-10-16T08:17:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a8069e90 0x7f15a80cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a806a270 0x7f15a81f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81f82c0 con 0x7f15a80c26b0 2023-10-16T08:17:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8158900 con 0x7f15a8069e90 2023-10-16T08:17:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a815a170 con 0x7f15a806a270 2023-10-16T08:17:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a8159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59330/0 (socket says 172.21.15.32:59330) 2023-10-16T08:17:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 -- 172.21.15.32:0/4279254284 learned_addr learned my addr 172.21.15.32:0/4279254284 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15adb94640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a8069e90 0x7f15a80cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ad393640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a806a270 0x7f15a81f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 -- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a806a270 msgr2=0x7f15a81f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a806a270 0x7f15a81f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 -- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a8069e90 msgr2=0x7f15a80cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a8069e90 0x7f15a80cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 -- 172.21.15.32:0/4279254284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009b30 con 0x7f15a80c26b0 2023-10-16T08:17:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15ae395640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a8159070 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f15a00af040 tx=0x7f15a0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00b8680 con 0x7f15a80c26b0 2023-10-16T08:17:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a00b87e0 con 0x7f15a80c26b0 2023-10-16T08:17:34.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00b8b90 con 0x7f15a80c26b0 2023-10-16T08:17:34.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a815a450 con 0x7f15a80c26b0 2023-10-16T08:17:34.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.181+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a815b510 con 0x7f15a80c26b0 2023-10-16T08:17:34.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.185+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a815a640 con 0x7f15a80c26b0 2023-10-16T08:17:34.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.185+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f15a00b8cf0 con 0x7f15a80c26b0 2023-10-16T08:17:34.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.185+0000 7f159e7fc640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f158c0664a0 0x7f158c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.185+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f15a011f750 con 0x7f15a80c26b0 2023-10-16T08:17:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.185+0000 7f15adb94640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f158c0664a0 0x7f158c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.185+0000 7f15adb94640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f158c0664a0 0x7f158c068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15a4007a60 tx=0x7f15a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.189+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f15a00ec9c0 con 0x7f15a80c26b0 2023-10-16T08:17:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.345+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15a804e170 con 0x7f15a80c26b0 2023-10-16T08:17:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.345+0000 7f159e7fc640 1 -- 172.21.15.32:0/4279254284 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15a00effe0 con 0x7f15a80c26b0 2023-10-16T08:17:34.353 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:34.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f158c0664a0 msgr2=0x7f158c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f158c0664a0 0x7f158c068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15a4007a60 tx=0x7f15a4008040 comp rx=0 tx=0).stop 2023-10-16T08:17:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 msgr2=0x7f15a8159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a8159070 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f15a00af040 tx=0x7f15a0019740 comp rx=0 tx=0).stop 2023-10-16T08:17:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 shutdown_connections 2023-10-16T08:17:34.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15a80c26b0 0x7f15a8159070 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f158c0664a0 0x7f158c068960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f15a806a270 0x7f15a81f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 --2- 172.21.15.32:0/4279254284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15a8069e90 0x7f15a80cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 >> 172.21.15.32:0/4279254284 conn(0x7f15a80b0da0 msgr2=0x7f15a80af500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:34.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 shutdown_connections 2023-10-16T08:17:34.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.349+0000 7f15afdf8640 1 -- 172.21.15.32:0/4279254284 wait complete. 2023-10-16T08:17:34.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:34.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.513+0000 7ff8849a7640 1 Processor -- start 2023-10-16T08:17:34.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.513+0000 7ff8849a7640 1 -- start start 2023-10-16T08:17:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff8849a7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff8800cf210 0x7ff8800cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff8849a7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff8800b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff8849a7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8800d3320 0x7ff8800b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff8849a7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8800cfb30 con 0x7ff8800d3320 2023-10-16T08:17:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff8849a7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8800d0880 con 0x7ff8800cf210 2023-10-16T08:17:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff8849a7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800b1c00 con 0x7ff88015cdc0 2023-10-16T08:17:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8800d3320 0x7ff8800b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8800d3320 0x7ff8800b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59346/0 (socket says 172.21.15.32:59346) 2023-10-16T08:17:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 -- 172.21.15.32:0/2289392304 learned_addr learned my addr 172.21.15.32:0/2289392304 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87f7fe640 1 --2- 172.21.15.32:0/2289392304 >> v2:172.21.15.186:3300/0 conn(0x7ff8800cf210 0x7ff8800cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87effd640 1 --2- 172.21.15.32:0/2289392304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff8800b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 -- 172.21.15.32:0/2289392304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 msgr2=0x7ff8800b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 --2- 172.21.15.32:0/2289392304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff8800b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 -- 172.21.15.32:0/2289392304 >> v2:172.21.15.186:3300/0 conn(0x7ff8800cf210 msgr2=0x7ff8800cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 --2- 172.21.15.32:0/2289392304 >> v2:172.21.15.186:3300/0 conn(0x7ff8800cf210 0x7ff8800cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.517+0000 7ff87ffff640 1 -- 172.21.15.32:0/2289392304 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800b1ee0 con 0x7ff8800d3320 2023-10-16T08:17:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff87ffff640 1 --2- 172.21.15.32:0/2289392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800b31e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff88008b880 tx=0x7ff87801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fccd7da03ab96ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff87e7fc640 1 -- 172.21.15.32:0/2289392304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780a5820 con 0x7ff8800d3320 2023-10-16T08:17:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff87e7fc640 1 -- 172.21.15.32:0/2289392304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8780a5980 con 0x7ff8800d3320 2023-10-16T08:17:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff87e7fc640 1 -- 172.21.15.32:0/2289392304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780a5d10 con 0x7ff8800d3320 2023-10-16T08:17:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 -- 172.21.15.32:0/2289392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 msgr2=0x7ff8800b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- 172.21.15.32:0/2289392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800b31e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff88008b880 tx=0x7ff87801c9a0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 -- 172.21.15.32:0/2289392304 shutdown_connections 2023-10-16T08:17:34.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- 172.21.15.32:0/2289392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800b31e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- 172.21.15.32:0/2289392304 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff8800b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- 172.21.15.32:0/2289392304 >> v2:172.21.15.186:3300/0 conn(0x7ff8800cf210 0x7ff8800cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 -- 172.21.15.32:0/2289392304 >> 172.21.15.32:0/2289392304 conn(0x7ff8800d2800 msgr2=0x7ff8800d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 -- 172.21.15.32:0/2289392304 shutdown_connections 2023-10-16T08:17:34.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 -- 172.21.15.32:0/2289392304 wait complete. 2023-10-16T08:17:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 Processor -- start 2023-10-16T08:17:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 -- start start 2023-10-16T08:17:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8800cf210 0x7ff8800caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.521+0000 7ff8849a7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff88014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff8849a7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88014fb20 con 0x7ff8800d3320 2023-10-16T08:17:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff8849a7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88015acd0 con 0x7ff8800cf210 2023-10-16T08:17:34.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff8849a7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff880153b00 con 0x7ff88015cdc0 2023-10-16T08:17:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59348/0 (socket says 172.21.15.32:59348) 2023-10-16T08:17:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 -- 172.21.15.32:0/913518998 learned_addr learned my addr 172.21.15.32:0/913518998 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87f7fe640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8800cf210 0x7ff8800caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87ffff640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff88014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 -- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 msgr2=0x7ff88014f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff88014f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 -- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8800cf210 msgr2=0x7ff8800caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8800cf210 0x7ff8800caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 -- 172.21.15.32:0/913518998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878009b30 con 0x7ff8800d3320 2023-10-16T08:17:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff87effd640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800cbf20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff87400d5f0 tx=0x7ff87400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8740125f0 con 0x7ff8800d3320 2023-10-16T08:17:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874012750 con 0x7ff8800d3320 2023-10-16T08:17:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff880153210 con 0x7ff8800d3320 2023-10-16T08:17:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874012ae0 con 0x7ff8800d3320 2023-10-16T08:17:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8801534b0 con 0x7ff8800d3320 2023-10-16T08:17:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88004e9c0 con 0x7ff8800d3320 2023-10-16T08:17:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.525+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff874012c40 con 0x7ff8800d3320 2023-10-16T08:17:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.529+0000 7ff867fff640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff86c0664a0 0x7ff86c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.529+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff8740846c0 con 0x7ff8800d3320 2023-10-16T08:17:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.529+0000 7ff87f7fe640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff86c0664a0 0x7ff86c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.529+0000 7ff87f7fe640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff86c0664a0 0x7ff86c068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff870006fd0 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.533+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff874051790 con 0x7ff8800d3320 2023-10-16T08:17:34.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.689+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8800c5a70 con 0x7ff8800d3320 2023-10-16T08:17:34.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.689+0000 7ff867fff640 1 -- 172.21.15.32:0/913518998 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff874007380 con 0x7ff8800d3320 2023-10-16T08:17:34.697 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff86c0664a0 msgr2=0x7ff86c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff86c0664a0 0x7ff86c068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff870006fd0 tx=0x7ff870008040 comp rx=0 tx=0).stop 2023-10-16T08:17:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 msgr2=0x7ff8800cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800cbf20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff87400d5f0 tx=0x7ff87400dab0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 shutdown_connections 2023-10-16T08:17:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff86c0664a0 0x7ff86c068960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff88015cdc0 0x7ff88014f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8800d3320 0x7ff8800cbf20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 --2- 172.21.15.32:0/913518998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8800cf210 0x7ff8800caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 >> 172.21.15.32:0/913518998 conn(0x7ff8800d2800 msgr2=0x7ff880158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:34.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 shutdown_connections 2023-10-16T08:17:34.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.693+0000 7ff8849a7640 1 -- 172.21.15.32:0/913518998 wait complete. 2023-10-16T08:17:34.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.857+0000 7fcaf411d640 1 Processor -- start 2023-10-16T08:17:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.857+0000 7fcaf411d640 1 -- start start 2023-10-16T08:17:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf411d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcaec159900 0x7fcaec06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf411d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec06a5a0 0x7fcaec0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf411d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaec0b4390 con 0x7fcaec0b3f00 2023-10-16T08:17:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf411d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec0cd740 con 0x7fcaec159900 2023-10-16T08:17:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf411d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec0b4600 con 0x7fcaec06a5a0 2023-10-16T08:17:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf1eb9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcaec159900 0x7fcaec06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf16b8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcaec0b3f00 0x7fcaec069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42196/0 (socket says 172.21.15.32:42196) 2023-10-16T08:17:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf16b8640 1 -- 172.21.15.32:0/1235166342 learned_addr learned my addr 172.21.15.32:0/1235166342 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf26ba640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec06a5a0 0x7fcaec0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/1235166342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec06a5a0 msgr2=0x7fcaec0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf1eb9640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec06a5a0 0x7fcaec0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/1235166342 >> v1:172.21.15.32:6789/0 conn(0x7fcaec0b3f00 legacy=0x7fcaec069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/1235166342 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0c8dd0 con 0x7fcaec159900 2023-10-16T08:17:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf26ba640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec06a5a0 0x7fcaec0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf0eb7640 1 -- 172.21.15.32:0/1235166342 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596388124 0 0) 0x7fcaec0b4390 con 0x7fcaec0b3f00 2023-10-16T08:17:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.861+0000 7fcaf1eb9640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec159900 0x7fcaec06e260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcae80097c0 tx=0x7fcae801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945b7ac63247749a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf0eb7640 1 -- 172.21.15.32:0/1235166342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80ab650 con 0x7fcaec159900 2023-10-16T08:17:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf0eb7640 1 -- 172.21.15.32:0/1235166342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae80ab7b0 con 0x7fcaec159900 2023-10-16T08:17:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf0eb7640 1 -- 172.21.15.32:0/1235166342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80abb40 con 0x7fcaec159900 2023-10-16T08:17:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- 172.21.15.32:0/1235166342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec159900 msgr2=0x7fcaec06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec159900 0x7fcaec06e260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcae80097c0 tx=0x7fcae801c450 comp rx=0 tx=0).stop 2023-10-16T08:17:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- 172.21.15.32:0/1235166342 shutdown_connections 2023-10-16T08:17:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec06a5a0 0x7fcaec0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 --2- 172.21.15.32:0/1235166342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec159900 0x7fcaec06e260 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- 172.21.15.32:0/1235166342 >> 172.21.15.32:0/1235166342 conn(0x7fcaec0b0da0 msgr2=0x7fcaec0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- 172.21.15.32:0/1235166342 shutdown_connections 2023-10-16T08:17:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- 172.21.15.32:0/1235166342 wait complete. 2023-10-16T08:17:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 Processor -- start 2023-10-16T08:17:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- start start 2023-10-16T08:17:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 0x7fcaec1519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaec0b3f00 0x7fcaec0acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec159900 0x7fcaec0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec152ef0 con 0x7fcaec06a5a0 2023-10-16T08:17:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1522e0 con 0x7fcaec0b3f00 2023-10-16T08:17:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf411d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1525c0 con 0x7fcaec159900 2023-10-16T08:17:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf1eb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 0x7fcaec1519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf1eb9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 0x7fcaec1519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59352/0 (socket says 172.21.15.32:59352) 2023-10-16T08:17:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.865+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/500140552 learned_addr learned my addr 172.21.15.32:0/500140552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf16b8640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaec0b3f00 0x7fcaec0acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec159900 msgr2=0x7fcaec0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf1eb9640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec159900 0x7fcaec0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaec0b3f00 msgr2=0x7fcaec0acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf1eb9640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaec0b3f00 0x7fcaec0acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf1eb9640 1 -- 172.21.15.32:0/500140552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae8009420 con 0x7fcaec06a5a0 2023-10-16T08:17:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf16b8640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaec0b3f00 0x7fcaec0acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf1eb9640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 0x7fcaec1519d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcae80ad040 tx=0x7fcae801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80b5810 con 0x7fcaec06a5a0 2023-10-16T08:17:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae80b5970 con 0x7fcaec06a5a0 2023-10-16T08:17:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80b5d20 con 0x7fcaec06a5a0 2023-10-16T08:17:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec1528a0 con 0x7fcaec06a5a0 2023-10-16T08:17:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec151f10 con 0x7fcaec06a5a0 2023-10-16T08:17:34.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec1520a0 con 0x7fcaec06a5a0 2023-10-16T08:17:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcae80bf410 con 0x7fcaec06a5a0 2023-10-16T08:17:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcae27fc640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcad00664a0 0x7fcad0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.869+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcae811efe0 con 0x7fcaec06a5a0 2023-10-16T08:17:34.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.873+0000 7fcaf16b8640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcad00664a0 0x7fcad0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:34.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.873+0000 7fcaf16b8640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcad00664a0 0x7fcad0068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcaec1f8340 tx=0x7fcadc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:34.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:34.873+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcaec1520a0 con 0x7fcaec06a5a0 2023-10-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.037+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcaec1520a0 con 0x7fcaec06a5a0 2023-10-16T08:17:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.037+0000 7fcae27fc640 1 -- 172.21.15.32:0/500140552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcaec1520a0 con 0x7fcaec06a5a0 2023-10-16T08:17:35.046 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:35.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcad00664a0 msgr2=0x7fcad0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcad00664a0 0x7fcad0068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcaec1f8340 tx=0x7fcadc009040 comp rx=0 tx=0).stop 2023-10-16T08:17:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 msgr2=0x7fcaec1519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 0x7fcaec1519d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcae80ad040 tx=0x7fcae801ac90 comp rx=0 tx=0).stop 2023-10-16T08:17:35.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 shutdown_connections 2023-10-16T08:17:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcad00664a0 0x7fcad0068960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcaec159900 0x7fcaec0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaec0b3f00 0x7fcaec0acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 --2- 172.21.15.32:0/500140552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaec06a5a0 0x7fcaec1519d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 >> 172.21.15.32:0/500140552 conn(0x7fcaec0b0da0 msgr2=0x7fcaec0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:35.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 shutdown_connections 2023-10-16T08:17:35.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.041+0000 7fcaf411d640 1 -- 172.21.15.32:0/500140552 wait complete. 2023-10-16T08:17:35.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.205+0000 7f6aa6a46640 1 Processor -- start 2023-10-16T08:17:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.205+0000 7f6aa6a46640 1 -- start start 2023-10-16T08:17:35.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6aa6a46640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6aa015cba0 0x7f6aa00d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6aa6a46640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00d1670 con 0x7f6aa00d0210 2023-10-16T08:17:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6aa6a46640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00d1950 con 0x7f6aa00b84e0 2023-10-16T08:17:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6aa6a46640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa00d1460 con 0x7f6aa015cba0 2023-10-16T08:17:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6aa015cba0 0x7f6aa00d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6aa015cba0 0x7f6aa00d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59364/0 (socket says 172.21.15.32:59364) 2023-10-16T08:17:35.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 -- 172.21.15.32:0/404365222 learned_addr learned my addr 172.21.15.32:0/404365222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:35.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6aa4fe3640 1 --1- 172.21.15.32:0/404365222 >> v1:172.21.15.32:6789/0 conn(0x7f6aa00d0210 0x7f6aa00b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42212/0 (socket says 172.21.15.32:42212) 2023-10-16T08:17:35.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 -- 172.21.15.32:0/404365222 >> v1:172.21.15.186:6789/0 conn(0x7f6aa00b84e0 legacy=0x7f6aa00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:35.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 -- 172.21.15.32:0/404365222 >> v1:172.21.15.32:6789/0 conn(0x7f6aa00d0210 legacy=0x7f6aa00b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 -- 172.21.15.32:0/404365222 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00d1c30 con 0x7f6aa015cba0 2023-10-16T08:17:35.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.209+0000 7f6a9ffff640 1 --2- 172.21.15.32:0/404365222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa015cba0 0x7f6aa00d0f20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009870 tx=0x7f6a9001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca3b9974f4cedcb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6a9effd640 1 -- 172.21.15.32:0/404365222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900ab670 con 0x7f6aa015cba0 2023-10-16T08:17:35.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6a9effd640 1 -- 172.21.15.32:0/404365222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a900ab7d0 con 0x7f6aa015cba0 2023-10-16T08:17:35.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6a9effd640 1 -- 172.21.15.32:0/404365222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900abb60 con 0x7f6aa015cba0 2023-10-16T08:17:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- 172.21.15.32:0/404365222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa015cba0 msgr2=0x7f6aa00d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/404365222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa015cba0 0x7f6aa00d0f20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009870 tx=0x7f6a9001c470 comp rx=0 tx=0).stop 2023-10-16T08:17:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- 172.21.15.32:0/404365222 shutdown_connections 2023-10-16T08:17:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/404365222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa015cba0 0x7f6aa00d0f20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- 172.21.15.32:0/404365222 >> 172.21.15.32:0/404365222 conn(0x7f6aa00719c0 msgr2=0x7f6aa0071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- 172.21.15.32:0/404365222 shutdown_connections 2023-10-16T08:17:35.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- 172.21.15.32:0/404365222 wait complete. 2023-10-16T08:17:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 Processor -- start 2023-10-16T08:17:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- start start 2023-10-16T08:17:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6aa00b84e0 0x7f6aa014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 0x7f6aa0148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa015cba0 0x7f6aa015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01494d0 con 0x7f6aa00d0210 2023-10-16T08:17:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa01504b0 con 0x7f6aa015cba0 2023-10-16T08:17:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6aa6a46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00c94e0 con 0x7f6aa00b84e0 2023-10-16T08:17:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6a9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 0x7f6aa0148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6a9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 0x7f6aa0148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59378/0 (socket says 172.21.15.32:59378) 2023-10-16T08:17:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.213+0000 7f6a9f7fe640 1 -- 172.21.15.32:0/687504451 learned_addr learned my addr 172.21.15.32:0/687504451 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9ffff640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6aa00b84e0 0x7f6aa014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6aa4fe3640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa015cba0 0x7f6aa015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9f7fe640 1 -- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6aa00b84e0 msgr2=0x7f6aa014bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9f7fe640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6aa00b84e0 0x7f6aa014bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9f7fe640 1 -- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa015cba0 msgr2=0x7f6aa015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9f7fe640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa015cba0 0x7f6aa015a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9f7fe640 1 -- 172.21.15.32:0/687504451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900094d0 con 0x7f6aa00d0210 2023-10-16T08:17:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9f7fe640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 0x7f6aa0148f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400c480 tx=0x7f6a9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94012830 con 0x7f6aa00d0210 2023-10-16T08:17:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a940129b0 con 0x7f6aa00d0210 2023-10-16T08:17:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa00c8cc0 con 0x7f6aa00d0210 2023-10-16T08:17:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94012d60 con 0x7f6aa00d0210 2023-10-16T08:17:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa00c9200 con 0x7f6aa00d0210 2023-10-16T08:17:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa0150640 con 0x7f6aa00d0210 2023-10-16T08:17:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6a9401c460 con 0x7f6aa00d0210 2023-10-16T08:17:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9cff9640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6a8c0664a0 0x7f6a8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.217+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6a94083f60 con 0x7f6aa00d0210 2023-10-16T08:17:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.221+0000 7f6a9ffff640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6a8c0664a0 0x7f6a8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.221+0000 7f6a9ffff640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6a8c0664a0 0x7f6a8c068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6a900af000 tx=0x7f6a900b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.221+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6a94051160 con 0x7f6aa00d0210 2023-10-16T08:17:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.377+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6aa004e170 con 0x7f6aa00d0210 2023-10-16T08:17:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.377+0000 7f6a9cff9640 1 -- 172.21.15.32:0/687504451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6a94054780 con 0x7f6aa00d0210 2023-10-16T08:17:35.385 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:35.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6a8c0664a0 msgr2=0x7f6a8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6a8c0664a0 0x7f6a8c068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6a900af000 tx=0x7f6a900b6040 comp rx=0 tx=0).stop 2023-10-16T08:17:35.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 msgr2=0x7f6aa0148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 0x7f6aa0148f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400c480 tx=0x7f6a9400c940 comp rx=0 tx=0).stop 2023-10-16T08:17:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 shutdown_connections 2023-10-16T08:17:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6aa015cba0 0x7f6aa015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6aa00d0210 0x7f6aa0148f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6a8c0664a0 0x7f6a8c068960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 --2- 172.21.15.32:0/687504451 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6aa00b84e0 0x7f6aa014bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 >> 172.21.15.32:0/687504451 conn(0x7f6aa00719c0 msgr2=0x7f6aa0155560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 shutdown_connections 2023-10-16T08:17:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.381+0000 7f6aa6a46640 1 -- 172.21.15.32:0/687504451 wait complete. 2023-10-16T08:17:35.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 Processor -- start 2023-10-16T08:17:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 -- start start 2023-10-16T08:17:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27641597c0 0x7f27640ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2764159ba0 0x7f27640cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f27640adc10 0x7f27640cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27640723d0 con 0x7f2764159ba0 2023-10-16T08:17:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27640717c0 con 0x7f27640adc10 2023-10-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f276ae6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2764071aa0 con 0x7f27641597c0 2023-10-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2764159ba0 0x7f27640cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2764159ba0 0x7f27640cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59384/0 (socket says 172.21.15.32:59384) 2023-10-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 -- 172.21.15.32:0/2996477406 learned_addr learned my addr 172.21.15.32:0/2996477406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2768c07640 1 --2- 172.21.15.32:0/2996477406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27641597c0 0x7f27640ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2769408640 1 --2- 172.21.15.32:0/2996477406 >> v2:172.21.15.186:3300/0 conn(0x7f27640adc10 0x7f27640cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 -- 172.21.15.32:0/2996477406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27641597c0 msgr2=0x7f27640ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 --2- 172.21.15.32:0/2996477406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27641597c0 0x7f27640ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 -- 172.21.15.32:0/2996477406 >> v2:172.21.15.186:3300/0 conn(0x7f27640adc10 msgr2=0x7f27640cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:35.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 --2- 172.21.15.32:0/2996477406 >> v2:172.21.15.186:3300/0 conn(0x7f27640adc10 0x7f27640cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.545+0000 7f2763fff640 1 -- 172.21.15.32:0/2996477406 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764071d80 con 0x7f2764159ba0 2023-10-16T08:17:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f2763fff640 1 --2- 172.21.15.32:0/2996477406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2764159ba0 0x7f27640cae10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f276406e330 tx=0x7f275001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f92068039b485f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f27637fe640 1 -- 172.21.15.32:0/2996477406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500a7070 con 0x7f2764159ba0 2023-10-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f27637fe640 1 -- 172.21.15.32:0/2996477406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500ab430 con 0x7f2764159ba0 2023-10-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f27637fe640 1 -- 172.21.15.32:0/2996477406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500ab7c0 con 0x7f2764159ba0 2023-10-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 -- 172.21.15.32:0/2996477406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2764159ba0 msgr2=0x7f27640cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 --2- 172.21.15.32:0/2996477406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2764159ba0 0x7f27640cae10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f276406e330 tx=0x7f275001c4d0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 -- 172.21.15.32:0/2996477406 shutdown_connections 2023-10-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 --2- 172.21.15.32:0/2996477406 >> v2:172.21.15.186:3300/0 conn(0x7f27640adc10 0x7f27640cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 --2- 172.21.15.32:0/2996477406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2764159ba0 0x7f27640cae10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 --2- 172.21.15.32:0/2996477406 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27641597c0 0x7f27640ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 -- 172.21.15.32:0/2996477406 >> 172.21.15.32:0/2996477406 conn(0x7f276406c190 msgr2=0x7f276406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 -- 172.21.15.32:0/2996477406 shutdown_connections 2023-10-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.549+0000 7f276ae6b640 1 -- 172.21.15.32:0/2996477406 wait complete. 2023-10-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 Processor -- start 2023-10-16T08:17:35.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 -- start start 2023-10-16T08:17:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 0x7f2764157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27641597c0 0x7f276414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2764159ba0 0x7f27640ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276414f8e0 con 0x7f27640adc10 2023-10-16T08:17:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276414fbc0 con 0x7f27641597c0 2023-10-16T08:17:35.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276414fea0 con 0x7f2764159ba0 2023-10-16T08:17:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 0x7f2764157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 0x7f2764157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59396/0 (socket says 172.21.15.32:59396) 2023-10-16T08:17:35.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 -- 172.21.15.32:0/1023678749 learned_addr learned my addr 172.21.15.32:0/1023678749 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2769408640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2764159ba0 0x7f27640ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2763fff640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27641597c0 0x7f276414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 -- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2764159ba0 msgr2=0x7f27640ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2764159ba0 0x7f27640ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 -- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27641597c0 msgr2=0x7f276414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27641597c0 0x7f276414f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 -- 172.21.15.32:0/1023678749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500094d0 con 0x7f27640adc10 2023-10-16T08:17:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2769408640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2764159ba0 0x7f27640ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2763fff640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27641597c0 0x7f276414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:35.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f2768c07640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 0x7f2764157bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f275800d800 tx=0x7f275800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275800b6c0 con 0x7f27640adc10 2023-10-16T08:17:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275800b820 con 0x7f27640adc10 2023-10-16T08:17:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275800bb90 con 0x7f27640adc10 2023-10-16T08:17:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2764150180 con 0x7f27640adc10 2023-10-16T08:17:35.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.553+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276414f5f0 con 0x7f27640adc10 2023-10-16T08:17:35.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.557+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2764074010 con 0x7f27640adc10 2023-10-16T08:17:35.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.557+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f275800bcf0 con 0x7f27640adc10 2023-10-16T08:17:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.557+0000 7f27617fa640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27480664a0 0x7f2748068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.557+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27580847c0 con 0x7f27640adc10 2023-10-16T08:17:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.557+0000 7f2763fff640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27480664a0 0x7f2748068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.557+0000 7f2763fff640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27480664a0 0x7f2748068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f27500af000 tx=0x7f2750008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.561+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2758051870 con 0x7f27640adc10 2023-10-16T08:17:35.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.729+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27640cd3f0 con 0x7f27640adc10 2023-10-16T08:17:35.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.729+0000 7f27617fa640 1 -- 172.21.15.32:0/1023678749 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2758054e90 con 0x7f27640adc10 2023-10-16T08:17:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:17:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27480664a0 msgr2=0x7f2748068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27480664a0 0x7f2748068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f27500af000 tx=0x7f2750008040 comp rx=0 tx=0).stop 2023-10-16T08:17:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 msgr2=0x7f2764157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 0x7f2764157bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f275800d800 tx=0x7f275800dcc0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 shutdown_connections 2023-10-16T08:17:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27480664a0 0x7f2748068960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2764159ba0 0x7f27640ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27641597c0 0x7f276414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 --2- 172.21.15.32:0/1023678749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27640adc10 0x7f2764157bf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 >> 172.21.15.32:0/1023678749 conn(0x7f276406c190 msgr2=0x7f2764151500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 shutdown_connections 2023-10-16T08:17:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.733+0000 7f276ae6b640 1 -- 172.21.15.32:0/1023678749 wait complete. 2023-10-16T08:17:35.759 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi186.front.sepia.ceph.com... 2023-10-16T08:17:35.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:35.760 DEBUG:teuthology.orchestra.run.smithi186:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-16T08:17:35.784 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:35.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:17:35.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:35.785 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T08:17:35.836 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T08:17:35.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:17:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.973+0000 7fd38e817640 1 Processor -- start 2023-10-16T08:17:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.973+0000 7fd38e817640 1 -- start start 2023-10-16T08:17:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.973+0000 7fd38e817640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3881597c0 0x7fd3880ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.973+0000 7fd38e817640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880723d0 con 0x7fd388159ba0 2023-10-16T08:17:35.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd38e817640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880717c0 con 0x7fd3880adc10 2023-10-16T08:17:35.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd38e817640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd388071aa0 con 0x7fd3881597c0 2023-10-16T08:17:35.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3881597c0 0x7fd3880ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3881597c0 0x7fd3880ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59402/0 (socket says 172.21.15.32:59402) 2023-10-16T08:17:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 -- 172.21.15.32:0/2188590420 learned_addr learned my addr 172.21.15.32:0/2188590420 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 -- 172.21.15.32:0/2188590420 >> v1:172.21.15.186:6789/0 conn(0x7fd3880adc10 legacy=0x7fd38815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 -- 172.21.15.32:0/2188590420 >> v1:172.21.15.32:6789/0 conn(0x7fd388159ba0 legacy=0x7fd3880cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 -- 172.21.15.32:0/2188590420 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388071d80 con 0x7fd3881597c0 2023-10-16T08:17:35.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd387fff640 1 --2- 172.21.15.32:0/2188590420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3880ad6d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd378009870 tx=0x7fd37801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1ed764c72560f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd386ffd640 1 -- 172.21.15.32:0/2188590420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780ab670 con 0x7fd3881597c0 2023-10-16T08:17:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd386ffd640 1 -- 172.21.15.32:0/2188590420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780ab7d0 con 0x7fd3881597c0 2023-10-16T08:17:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.977+0000 7fd386ffd640 1 -- 172.21.15.32:0/2188590420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780abb60 con 0x7fd3881597c0 2023-10-16T08:17:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- 172.21.15.32:0/2188590420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 msgr2=0x7fd3880ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 --2- 172.21.15.32:0/2188590420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3880ad6d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd378009870 tx=0x7fd37801c4d0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- 172.21.15.32:0/2188590420 shutdown_connections 2023-10-16T08:17:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 --2- 172.21.15.32:0/2188590420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3880ad6d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- 172.21.15.32:0/2188590420 >> 172.21.15.32:0/2188590420 conn(0x7fd38806c190 msgr2=0x7fd38806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- 172.21.15.32:0/2188590420 shutdown_connections 2023-10-16T08:17:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- 172.21.15.32:0/2188590420 wait complete. 2023-10-16T08:17:35.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 Processor -- start 2023-10-16T08:17:35.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- start start 2023-10-16T08:17:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3880adc10 0x7fd3881fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3881f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd388159ba0 0x7fd38815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880c6320 con 0x7fd3881597c0 2023-10-16T08:17:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881fd280 con 0x7fd388159ba0 2023-10-16T08:17:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38e817640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880c5730 con 0x7fd3880adc10 2023-10-16T08:17:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3881f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3881f9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59418/0 (socket says 172.21.15.32:59418) 2023-10-16T08:17:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd3877fe640 1 -- 172.21.15.32:0/360403242 learned_addr learned my addr 172.21.15.32:0/360403242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd387fff640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3880adc10 0x7fd3881fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.981+0000 7fd38cdb4640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd388159ba0 0x7fd38815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd3877fe640 1 -- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3880adc10 msgr2=0x7fd3881fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd3877fe640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3880adc10 0x7fd3881fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd3877fe640 1 -- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd388159ba0 msgr2=0x7fd38815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:35.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd3877fe640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd388159ba0 0x7fd38815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:35.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd3877fe640 1 -- 172.21.15.32:0/360403242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780094d0 con 0x7fd3881597c0 2023-10-16T08:17:35.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd38cdb4640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd388159ba0 0x7fd38815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:35.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd3877fe640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3881f9f50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00c3c0 tx=0x7fd37c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:35.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0052d0 con 0x7fd3881597c0 2023-10-16T08:17:35.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c005430 con 0x7fd3881597c0 2023-10-16T08:17:35.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c003840 con 0x7fd3881597c0 2023-10-16T08:17:35.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3880c59f0 con 0x7fd3881597c0 2023-10-16T08:17:35.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3880c5fb0 con 0x7fd3881597c0 2023-10-16T08:17:35.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd388074010 con 0x7fd3881597c0 2023-10-16T08:17:35.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd37c0039a0 con 0x7fd3881597c0 2023-10-16T08:17:35.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd384ff9640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3740664a0 0x7fd374068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:35.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd37c083790 con 0x7fd3881597c0 2023-10-16T08:17:35.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.985+0000 7fd387fff640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3740664a0 0x7fd374068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:35.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.989+0000 7fd387fff640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3740664a0 0x7fd374068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd3780af000 tx=0x7fd3780b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:35.989+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd37c050860 con 0x7fd3881597c0 2023-10-16T08:17:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3880cd3f0 con 0x7fd3881597c0 2023-10-16T08:17:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd384ff9640 1 -- 172.21.15.32:0/360403242 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7fd37c053e80 con 0x7fd3881597c0 2023-10-16T08:17:36.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:36.192 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:17:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:17:36.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3740664a0 msgr2=0x7fd374068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3740664a0 0x7fd374068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd3780af000 tx=0x7fd3780b6040 comp rx=0 tx=0).stop 2023-10-16T08:17:36.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 msgr2=0x7fd3881f9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3881f9f50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00c3c0 tx=0x7fd37c00c880 comp rx=0 tx=0).stop 2023-10-16T08:17:36.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 shutdown_connections 2023-10-16T08:17:36.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd388159ba0 0x7fd38815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3881597c0 0x7fd3881f9f50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3740664a0 0x7fd374068960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 --2- 172.21.15.32:0/360403242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd3880adc10 0x7fd3881fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 >> 172.21.15.32:0/360403242 conn(0x7fd38806c190 msgr2=0x7fd3880cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:36.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 shutdown_connections 2023-10-16T08:17:36.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.185+0000 7fd38e817640 1 -- 172.21.15.32:0/360403242 wait complete. 2023-10-16T08:17:36.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.349+0000 7fd800fd0640 1 Processor -- start 2023-10-16T08:17:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.349+0000 7fd800fd0640 1 -- start start 2023-10-16T08:17:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd800fd0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd7fc1597c0 0x7fd7fc0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd800fd0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7fc0723d0 con 0x7fd7fc0adc10 2023-10-16T08:17:36.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd800fd0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7fc0717c0 con 0x7fd7fc159ba0 2023-10-16T08:17:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd800fd0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7fc071aa0 con 0x7fd7fc1597c0 2023-10-16T08:17:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7fad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd7fc0adc10 0x7fd7fc15d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42224/0 (socket says 172.21.15.32:42224) 2023-10-16T08:17:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7fad9d640 1 -- 172.21.15.32:0/2719724040 learned_addr learned my addr 172.21.15.32:0/2719724040 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:36.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7fa59c640 1 --2- 172.21.15.32:0/2719724040 >> v2:172.21.15.186:3300/0 conn(0x7fd7fc1597c0 0x7fd7fc0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7f9d9b640 1 -- 172.21.15.32:0/2719724040 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169899746 0 0) 0x7fd7fc0723d0 con 0x7fd7fc0adc10 2023-10-16T08:17:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7f9d9b640 1 -- 172.21.15.32:0/2719724040 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e8005740 con 0x7fd7fc0adc10 2023-10-16T08:17:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7fa59c640 1 -- 172.21.15.32:0/2719724040 >> v1:172.21.15.186:6789/0 conn(0x7fd7fc159ba0 legacy=0x7fd7fc0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7fa59c640 1 -- 172.21.15.32:0/2719724040 >> v1:172.21.15.32:6789/0 conn(0x7fd7fc0adc10 legacy=0x7fd7fc15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.353+0000 7fd7fa59c640 1 -- 172.21.15.32:0/2719724040 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc071d80 con 0x7fd7fc1597c0 2023-10-16T08:17:36.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd7fa59c640 1 --2- 172.21.15.32:0/2719724040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0ad6d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0097c0 tx=0x7fd7ec01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17a143635a19bf27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd7f9d9b640 1 -- 172.21.15.32:0/2719724040 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0ab5f0 con 0x7fd7fc1597c0 2023-10-16T08:17:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd7f9d9b640 1 -- 172.21.15.32:0/2719724040 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec0ab750 con 0x7fd7fc1597c0 2023-10-16T08:17:36.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd7f9d9b640 1 -- 172.21.15.32:0/2719724040 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0abae0 con 0x7fd7fc1597c0 2023-10-16T08:17:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 -- 172.21.15.32:0/2719724040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 msgr2=0x7fd7fc0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 --2- 172.21.15.32:0/2719724040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0ad6d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0097c0 tx=0x7fd7ec01c450 comp rx=0 tx=0).stop 2023-10-16T08:17:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 -- 172.21.15.32:0/2719724040 shutdown_connections 2023-10-16T08:17:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 --2- 172.21.15.32:0/2719724040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0ad6d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 -- 172.21.15.32:0/2719724040 >> 172.21.15.32:0/2719724040 conn(0x7fd7fc06c190 msgr2=0x7fd7fc06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 -- 172.21.15.32:0/2719724040 shutdown_connections 2023-10-16T08:17:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 -- 172.21.15.32:0/2719724040 wait complete. 2023-10-16T08:17:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 Processor -- start 2023-10-16T08:17:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.357+0000 7fd800fd0640 1 -- start start 2023-10-16T08:17:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7fc0adc10 0x7fd7fc14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 0x7fd7fc1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc156e20 con 0x7fd7fc159ba0 2023-10-16T08:17:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc0b44d0 con 0x7fd7fc1597c0 2023-10-16T08:17:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc0b3c30 con 0x7fd7fc0adc10 2023-10-16T08:17:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 0x7fd7fc1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 0x7fd7fc1546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59422/0 (socket says 172.21.15.32:59422) 2023-10-16T08:17:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 -- 172.21.15.32:0/3685000998 learned_addr learned my addr 172.21.15.32:0/3685000998 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fa59c640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7fc0adc10 0x7fd7fc14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7f3fff640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 -- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7fc0adc10 msgr2=0x7fd7fc14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7fc0adc10 0x7fd7fc14f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 -- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 msgr2=0x7fd7fc0b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 -- 172.21.15.32:0/3685000998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec009420 con 0x7fd7fc159ba0 2023-10-16T08:17:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7fad9d640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 0x7fd7fc1546d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400caa0 tx=0x7fd7f400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400ac00 con 0x7fd7fc159ba0 2023-10-16T08:17:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f400ad60 con 0x7fd7fc159ba0 2023-10-16T08:17:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7fc0b3f10 con 0x7fd7fc159ba0 2023-10-16T08:17:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4012620 con 0x7fd7fc159ba0 2023-10-16T08:17:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7fc14f830 con 0x7fd7fc159ba0 2023-10-16T08:17:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7fc156f90 con 0x7fd7fc159ba0 2023-10-16T08:17:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.361+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd7f4007070 con 0x7fd7fc159ba0 2023-10-16T08:17:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.365+0000 7fd7f2ffd640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7dc0664d0 0x7fd7dc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.365+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd7f40841b0 con 0x7fd7fc159ba0 2023-10-16T08:17:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.365+0000 7fd7fa59c640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7dc0664d0 0x7fd7dc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.365+0000 7fd7fa59c640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7dc0664d0 0x7fd7dc068990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0af000 tx=0x7fd7ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.365+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd7f4051280 con 0x7fd7fc159ba0 2023-10-16T08:17:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.525+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7fc074010 con 0x7fd7fc159ba0 2023-10-16T08:17:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd7f2ffd640 1 -- 172.21.15.32:0/3685000998 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fd7f40548a0 con 0x7fd7fc159ba0 2023-10-16T08:17:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:36.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7dc0664d0 msgr2=0x7fd7dc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7dc0664d0 0x7fd7dc068990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0af000 tx=0x7fd7ec0a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 msgr2=0x7fd7fc1546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 0x7fd7fc1546d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400caa0 tx=0x7fd7f400cf60 comp rx=0 tx=0).stop 2023-10-16T08:17:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 shutdown_connections 2023-10-16T08:17:36.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7fc159ba0 0x7fd7fc1546d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7fc1597c0 0x7fd7fc0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7dc0664d0 0x7fd7dc068990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 --2- 172.21.15.32:0/3685000998 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd7fc0adc10 0x7fd7fc14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 >> 172.21.15.32:0/3685000998 conn(0x7fd7fc06c190 msgr2=0x7fd7fc15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 shutdown_connections 2023-10-16T08:17:36.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.529+0000 7fd800fd0640 1 -- 172.21.15.32:0/3685000998 wait complete. 2023-10-16T08:17:36.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:16:20.036424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:16:21.327248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T08:16:25.022345+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T08:17:36.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:36.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639bb9d640 1 Processor -- start 2023-10-16T08:17:36.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639bb9d640 1 -- start start 2023-10-16T08:17:36.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639bb9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c36c0 0x7f6394165570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639bb9d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639415ae30 con 0x7f639406d010 2023-10-16T08:17:36.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639bb9d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63940ca510 con 0x7f6394069e90 2023-10-16T08:17:36.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639bb9d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940cb090 con 0x7f63940c36c0 2023-10-16T08:17:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399138640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f639406d010 0x7f63940c9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42232/0 (socket says 172.21.15.32:42232) 2023-10-16T08:17:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399138640 1 -- 172.21.15.32:0/1782671003 learned_addr learned my addr 172.21.15.32:0/1782671003 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:36.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639a13a640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c36c0 0x7f6394165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111094014 0 0) 0x7f639415ae30 con 0x7f639406d010 2023-10-16T08:17:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6380003530 con 0x7f639406d010 2023-10-16T08:17:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2230518100 0 0) 0x7f6380003530 con 0x7f639406d010 2023-10-16T08:17:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c36c0 msgr2=0x7f6394165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c36c0 0x7f6394165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 >> v1:172.21.15.186:6789/0 conn(0x7f6394069e90 legacy=0x7f639406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940ca7f0 con 0x7f639406d010 2023-10-16T08:17:36.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f6384001680 con 0x7f639406d010 2023-10-16T08:17:36.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 >> v1:172.21.15.32:6789/0 conn(0x7f639406d010 legacy=0x7f63940c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63800039b0 0x7f6380003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6380004360 0x7f6380004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6380005030 0x7f638000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940cb090 con 0x7f63800039b0 2023-10-16T08:17:36.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638000bed0 con 0x7f6380004360 2023-10-16T08:17:36.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638000c010 con 0x7f6380005030 2023-10-16T08:17:36.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639a13a640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c36c0 0x7f6394165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:36.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399939640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63800039b0 0x7f6380003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399138640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6380004360 0x7f6380004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f639a13a640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6380005030 0x7f638000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399939640 1 -- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6380005030 msgr2=0x7f638000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399939640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6380005030 0x7f638000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399939640 1 -- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6380004360 msgr2=0x7f6380004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.697+0000 7f6399939640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6380004360 0x7f6380004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f6399939640 1 -- 172.21.15.32:0/1782671003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6380003530 con 0x7f63800039b0 2023-10-16T08:17:36.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f6399939640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63800039b0 0x7f6380003d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6390001860 tx=0x7f639001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d32aa447dcea87 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900a8890 con 0x7f63800039b0 2023-10-16T08:17:36.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900a89f0 con 0x7f63800039b0 2023-10-16T08:17:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f6398937640 1 -- 172.21.15.32:0/1782671003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900a8da0 con 0x7f63800039b0 2023-10-16T08:17:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639bb9d640 1 -- 172.21.15.32:0/1782671003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63800039b0 msgr2=0x7f6380003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639bb9d640 1 --2- 172.21.15.32:0/1782671003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63800039b0 0x7f6380003d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6390001860 tx=0x7f639001ec70 comp rx=0 tx=0).stop 2023-10-16T08:17:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639a13a640 1 -- 172.21.15.32:0/1782671003 reap_dead start 2023-10-16T08:17:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639bb9d640 1 -- 172.21.15.32:0/1782671003 shutdown_connections 2023-10-16T08:17:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639bb9d640 1 -- 172.21.15.32:0/1782671003 >> 172.21.15.32:0/1782671003 conn(0x7f63940b0da0 msgr2=0x7f63940ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:36.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639bb9d640 1 -- 172.21.15.32:0/1782671003 shutdown_connections 2023-10-16T08:17:36.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.701+0000 7f639bb9d640 1 -- 172.21.15.32:0/1782671003 wait complete. 2023-10-16T08:17:36.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 Processor -- start 2023-10-16T08:17:36.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 -- start start 2023-10-16T08:17:36.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63800039b0 0x7f6394152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c5bf0 0x7f639415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 0x7f639415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940c6c00 con 0x7f6394152fd0 2023-10-16T08:17:36.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639415be20 con 0x7f63800039b0 2023-10-16T08:17:36.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940c6080 con 0x7f63940c5bf0 2023-10-16T08:17:36.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 0x7f639415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 0x7f639415c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59440/0 (socket says 172.21.15.32:59440) 2023-10-16T08:17:36.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 -- 172.21.15.32:0/2696118940 learned_addr learned my addr 172.21.15.32:0/2696118940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:36.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f6399138640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c5bf0 0x7f639415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f6399939640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63800039b0 0x7f6394152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 -- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c5bf0 msgr2=0x7f639415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c5bf0 0x7f639415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 -- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63800039b0 msgr2=0x7f6394152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63800039b0 0x7f6394152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 -- 172.21.15.32:0/2696118940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940c6360 con 0x7f6394152fd0 2023-10-16T08:17:36.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f6399138640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c5bf0 0x7f639415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:36.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639a13a640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 0x7f639415c7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63940c2060 tx=0x7f638c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00e070 con 0x7f6394152fd0 2023-10-16T08:17:36.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c0039a0 con 0x7f6394152fd0 2023-10-16T08:17:36.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c003b30 con 0x7f6394152fd0 2023-10-16T08:17:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63940c6640 con 0x7f6394152fd0 2023-10-16T08:17:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.705+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63941539f0 con 0x7f6394152fd0 2023-10-16T08:17:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.709+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639415bf90 con 0x7f6394152fd0 2023-10-16T08:17:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.709+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f638c003c90 con 0x7f6394152fd0 2023-10-16T08:17:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.709+0000 7f638a7fc640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63780664d0 0x7f6378068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:36.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.709+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f638c084ab0 con 0x7f6394152fd0 2023-10-16T08:17:36.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.709+0000 7f6399939640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63780664d0 0x7f6378068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:36.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.713+0000 7f6399939640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63780664d0 0x7f6378068990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63900b2000 tx=0x7f63900aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:36.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.713+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f639415bf90 con 0x7f6394152fd0 2023-10-16T08:17:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.869+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f639415bf90 con 0x7f6394152fd0 2023-10-16T08:17:36.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f638a7fc640 1 -- 172.21.15.32:0/2696118940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f639415bf90 con 0x7f6394152fd0 2023-10-16T08:17:36.878 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63780664d0 msgr2=0x7f6378068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63780664d0 0x7f6378068990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63900b2000 tx=0x7f63900aa070 comp rx=0 tx=0).stop 2023-10-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 msgr2=0x7f639415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:36.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 0x7f639415c7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63940c2060 tx=0x7f638c011890 comp rx=0 tx=0).stop 2023-10-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 shutdown_connections 2023-10-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6394152fd0 0x7f639415c7b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63780664d0 0x7f6378068990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63940c5bf0 0x7f639415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 --2- 172.21.15.32:0/2696118940 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63800039b0 0x7f6394152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 >> 172.21.15.32:0/2696118940 conn(0x7f63940b0da0 msgr2=0x7f63940c45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 shutdown_connections 2023-10-16T08:17:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:36.873+0000 7f639bb9d640 1 -- 172.21.15.32:0/2696118940 wait complete. 2023-10-16T08:17:36.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:17:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 Processor -- start 2023-10-16T08:17:37.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 -- start start 2023-10-16T08:17:37.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd374159900 0x7fd37406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd37406a5a0 0x7fd3740c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3740b4390 con 0x7fd3740b3f00 2023-10-16T08:17:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3740cd740 con 0x7fd374159900 2023-10-16T08:17:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd37a5d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740b4600 con 0x7fd37406a5a0 2023-10-16T08:17:37.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd374159900 0x7fd37406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd374159900 0x7fd37406e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59456/0 (socket says 172.21.15.32:59456) 2023-10-16T08:17:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 -- 172.21.15.32:0/4091167997 learned_addr learned my addr 172.21.15.32:0/4091167997 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:37.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd378b76640 1 --2- 172.21.15.32:0/4091167997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd37406a5a0 0x7fd3740c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 -- 172.21.15.32:0/4091167997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd37406a5a0 msgr2=0x7fd3740c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 --2- 172.21.15.32:0/4091167997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd37406a5a0 0x7fd3740c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 -- 172.21.15.32:0/4091167997 >> v1:172.21.15.186:6789/0 conn(0x7fd3740b3f00 legacy=0x7fd374069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.041+0000 7fd373fff640 1 -- 172.21.15.32:0/4091167997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740c8dd0 con 0x7fd374159900 2023-10-16T08:17:37.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd373fff640 1 --2- 172.21.15.32:0/4091167997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd374159900 0x7fd37406e260 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd36400bb10 tx=0x7fd36401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af719fae5b913518 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd372ffd640 1 -- 172.21.15.32:0/4091167997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640a7cb0 con 0x7fd374159900 2023-10-16T08:17:37.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd372ffd640 1 -- 172.21.15.32:0/4091167997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3640a7e10 con 0x7fd374159900 2023-10-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd372ffd640 1 -- 172.21.15.32:0/4091167997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640af660 con 0x7fd374159900 2023-10-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 -- 172.21.15.32:0/4091167997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd374159900 msgr2=0x7fd37406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/4091167997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd374159900 0x7fd37406e260 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd36400bb10 tx=0x7fd36401ebe0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 -- 172.21.15.32:0/4091167997 shutdown_connections 2023-10-16T08:17:37.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/4091167997 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd37406a5a0 0x7fd3740c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/4091167997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd374159900 0x7fd37406e260 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 -- 172.21.15.32:0/4091167997 >> 172.21.15.32:0/4091167997 conn(0x7fd3740b0da0 msgr2=0x7fd3740ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 -- 172.21.15.32:0/4091167997 shutdown_connections 2023-10-16T08:17:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.045+0000 7fd37a5d9640 1 -- 172.21.15.32:0/4091167997 wait complete. 2023-10-16T08:17:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 Processor -- start 2023-10-16T08:17:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 -- start start 2023-10-16T08:17:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd37406a5a0 0x7fd3740c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 0x7fd3740ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd374159900 0x7fd374153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740c2450 con 0x7fd3740b3f00 2023-10-16T08:17:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740c26f0 con 0x7fd37406a5a0 2023-10-16T08:17:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740c29d0 con 0x7fd374159900 2023-10-16T08:17:37.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 0x7fd3740ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 0x7fd3740ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59458/0 (socket says 172.21.15.32:59458) 2023-10-16T08:17:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 -- 172.21.15.32:0/3183324872 learned_addr learned my addr 172.21.15.32:0/3183324872 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd373fff640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd37406a5a0 0x7fd3740c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd378b76640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd374159900 0x7fd374153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 -- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd374159900 msgr2=0x7fd374153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd374159900 0x7fd374153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 -- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd37406a5a0 msgr2=0x7fd3740c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd37406a5a0 0x7fd3740c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 -- 172.21.15.32:0/3183324872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36400b770 con 0x7fd3740b3f00 2023-10-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd3737fe640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 0x7fd3740ace20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c500 tx=0x7fd36800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368005220 con 0x7fd3740b3f00 2023-10-16T08:17:37.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368005380 con 0x7fd3740b3f00 2023-10-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3740c2cb0 con 0x7fd3740b3f00 2023-10-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368007270 con 0x7fd3740b3f00 2023-10-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.049+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd374158d70 con 0x7fd3740b3f00 2023-10-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.053+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37404e170 con 0x7fd3740b3f00 2023-10-16T08:17:37.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.053+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd3680073d0 con 0x7fd3740b3f00 2023-10-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.053+0000 7fd370ff9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3600664a0 0x7fd360068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.053+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd368083a90 con 0x7fd3740b3f00 2023-10-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.053+0000 7fd373fff640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3600664a0 0x7fd360068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.053+0000 7fd373fff640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3600664a0 0x7fd360068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd3640b1000 tx=0x7fd364008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.057+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd368050b70 con 0x7fd3740b3f00 2023-10-16T08:17:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.213+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3740ce940 con 0x7fd3740b3f00 2023-10-16T08:17:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.213+0000 7fd370ff9640 1 -- 172.21.15.32:0/3183324872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd368054190 con 0x7fd3740b3f00 2023-10-16T08:17:37.221 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3600664a0 msgr2=0x7fd360068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3600664a0 0x7fd360068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd3640b1000 tx=0x7fd364008040 comp rx=0 tx=0).stop 2023-10-16T08:17:37.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 msgr2=0x7fd3740ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 0x7fd3740ace20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c500 tx=0x7fd36800c9c0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 shutdown_connections 2023-10-16T08:17:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3600664a0 0x7fd360068960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd374159900 0x7fd374153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740b3f00 0x7fd3740ace20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 --2- 172.21.15.32:0/3183324872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd37406a5a0 0x7fd3740c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 >> 172.21.15.32:0/3183324872 conn(0x7fd3740b0da0 msgr2=0x7fd3740ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 shutdown_connections 2023-10-16T08:17:37.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.217+0000 7fd37a5d9640 1 -- 172.21.15.32:0/3183324872 wait complete. 2023-10-16T08:17:37.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:17:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 Processor -- start 2023-10-16T08:17:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 -- start start 2023-10-16T08:17:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef0069e90 0x7f4ef006d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ef00c26b0 0x7f4ef00c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef015ae30 con 0x7f4ef006a270 2023-10-16T08:17:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef00cb090 con 0x7f4ef00c26b0 2023-10-16T08:17:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.381+0000 7f4ef49d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00ca510 con 0x7f4ef0069e90 2023-10-16T08:17:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eeed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ef00c26b0 0x7f4ef00c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4ee7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ef006a270 0x7f4ef00c9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42236/0 (socket says 172.21.15.32:42236) 2023-10-16T08:17:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4ee7fff640 1 -- 172.21.15.32:0/3013420902 learned_addr learned my addr 172.21.15.32:0/3013420902 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eee59c640 1 --2- 172.21.15.32:0/3013420902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef0069e90 0x7f4ef006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eeed9d640 1 -- 172.21.15.32:0/3013420902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef0069e90 msgr2=0x7f4ef006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eeed9d640 1 --2- 172.21.15.32:0/3013420902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef0069e90 0x7f4ef006d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eeed9d640 1 -- 172.21.15.32:0/3013420902 >> v1:172.21.15.32:6789/0 conn(0x7f4ef006a270 legacy=0x7f4ef00c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eeed9d640 1 -- 172.21.15.32:0/3013420902 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00ca7f0 con 0x7f4ef00c26b0 2023-10-16T08:17:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eedd9b640 1 -- 172.21.15.32:0/3013420902 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400024612 0 0) 0x7f4ef015ae30 con 0x7f4ef006a270 2023-10-16T08:17:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eeed9d640 1 --2- 172.21.15.32:0/3013420902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef00c26b0 0x7f4ef00c2b40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009d90 tx=0x7f4ee801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a6680c355a8a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eedd9b640 1 -- 172.21.15.32:0/3013420902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80a59e0 con 0x7f4ef00c26b0 2023-10-16T08:17:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eedd9b640 1 -- 172.21.15.32:0/3013420902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee80a5b40 con 0x7f4ef00c26b0 2023-10-16T08:17:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.385+0000 7f4eedd9b640 1 -- 172.21.15.32:0/3013420902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80ad440 con 0x7f4ef00c26b0 2023-10-16T08:17:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- 172.21.15.32:0/3013420902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef00c26b0 msgr2=0x7f4ef00c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/3013420902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef00c26b0 0x7f4ef00c2b40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009d90 tx=0x7f4ee801c920 comp rx=0 tx=0).stop 2023-10-16T08:17:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- 172.21.15.32:0/3013420902 shutdown_connections 2023-10-16T08:17:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/3013420902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef00c26b0 0x7f4ef00c2b40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/3013420902 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef0069e90 0x7f4ef006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- 172.21.15.32:0/3013420902 >> 172.21.15.32:0/3013420902 conn(0x7f4ef00b0da0 msgr2=0x7f4ef00ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- 172.21.15.32:0/3013420902 shutdown_connections 2023-10-16T08:17:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- 172.21.15.32:0/3013420902 wait complete. 2023-10-16T08:17:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 Processor -- start 2023-10-16T08:17:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- start start 2023-10-16T08:17:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 0x7f4ef015d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef006a270 0x7f4ef00ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef00c26b0 0x7f4ef00aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef014c4a0 con 0x7f4ef0069e90 2023-10-16T08:17:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00cc4f0 con 0x7f4ef006a270 2023-10-16T08:17:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ef49d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef015bdd0 con 0x7f4ef00c26b0 2023-10-16T08:17:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 0x7f4ef015d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 0x7f4ef015d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59476/0 (socket says 172.21.15.32:59476) 2023-10-16T08:17:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 -- 172.21.15.32:0/592105038 learned_addr learned my addr 172.21.15.32:0/592105038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eeed9d640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef00c26b0 0x7f4ef00aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4ee7fff640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef006a270 0x7f4ef00ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 -- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef00c26b0 msgr2=0x7f4ef00aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef00c26b0 0x7f4ef00aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 -- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef006a270 msgr2=0x7f4ef00ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef006a270 0x7f4ef00ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.389+0000 7f4eee59c640 1 -- 172.21.15.32:0/592105038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee8009a70 con 0x7f4ef0069e90 2023-10-16T08:17:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4eee59c640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 0x7f4ef015d8e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000e930 tx=0x7f4ee000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0009690 con 0x7f4ef0069e90 2023-10-16T08:17:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee000ca00 con 0x7f4ef0069e90 2023-10-16T08:17:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000cd70 con 0x7f4ef0069e90 2023-10-16T08:17:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef015de20 con 0x7f4ef0069e90 2023-10-16T08:17:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef00cca70 con 0x7f4ef0069e90 2023-10-16T08:17:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef004e170 con 0x7f4ef0069e90 2023-10-16T08:17:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ee0013470 con 0x7f4ef0069e90 2023-10-16T08:17:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee6ffd640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ed00664a0 0x7f4ed0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4ee0084510 con 0x7f4ef0069e90 2023-10-16T08:17:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.393+0000 7f4ee7fff640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ed00664a0 0x7f4ed0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.397+0000 7f4ee7fff640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ed00664a0 0x7f4ed0068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ca150 tx=0x7f4ed8007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.397+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ee00515f0 con 0x7f4ef0069e90 2023-10-16T08:17:37.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ef00c2090 con 0x7f4ef0069e90 2023-10-16T08:17:37.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ee6ffd640 1 -- 172.21.15.32:0/592105038 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ee0054c10 con 0x7f4ef0069e90 2023-10-16T08:17:37.559 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:17:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ed00664a0 msgr2=0x7f4ed0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ed00664a0 0x7f4ed0068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ca150 tx=0x7f4ed8007b60 comp rx=0 tx=0).stop 2023-10-16T08:17:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 msgr2=0x7f4ef015d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 0x7f4ef015d8e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000e930 tx=0x7f4ee000edf0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 shutdown_connections 2023-10-16T08:17:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ed00664a0 0x7f4ed0068960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4ef00c26b0 0x7f4ef00aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef006a270 0x7f4ef00ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 --2- 172.21.15.32:0/592105038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ef0069e90 0x7f4ef015d8e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 >> 172.21.15.32:0/592105038 conn(0x7f4ef00b0da0 msgr2=0x7f4ef01561e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 shutdown_connections 2023-10-16T08:17:37.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.553+0000 7f4ef49d9640 1 -- 172.21.15.32:0/592105038 wait complete. 2023-10-16T08:17:37.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T08:17:37.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.717+0000 7f7af3c91640 1 Processor -- start 2023-10-16T08:17:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af3c91640 1 -- start start 2023-10-16T08:17:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af3c91640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7aec069e90 0x7f7aec06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af3c91640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec15ae30 con 0x7f7aec06a270 2023-10-16T08:17:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af3c91640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec0ca510 con 0x7f7aec0c26b0 2023-10-16T08:17:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af3c91640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec0cb090 con 0x7f7aec069e90 2023-10-16T08:17:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af122c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7aec06a270 0x7f7aec0c9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42250/0 (socket says 172.21.15.32:42250) 2023-10-16T08:17:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af122c640 1 -- 172.21.15.32:0/1072351304 learned_addr learned my addr 172.21.15.32:0/1072351304 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af1a2d640 1 --2- 172.21.15.32:0/1072351304 >> v2:172.21.15.186:3300/0 conn(0x7f7aec069e90 0x7f7aec06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118972737 0 0) 0x7f7aec15ae30 con 0x7f7aec06a270 2023-10-16T08:17:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad8003530 con 0x7f7aec06a270 2023-10-16T08:17:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938405524 0 0) 0x7f7aec0ca510 con 0x7f7aec0c26b0 2023-10-16T08:17:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aec15ae30 con 0x7f7aec0c26b0 2023-10-16T08:17:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af1a2d640 1 -- 172.21.15.32:0/1072351304 >> v1:172.21.15.186:6789/0 conn(0x7f7aec0c26b0 legacy=0x7f7aec0c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af1a2d640 1 -- 172.21.15.32:0/1072351304 >> v1:172.21.15.32:6789/0 conn(0x7f7aec06a270 legacy=0x7f7aec0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.721+0000 7f7af1a2d640 1 -- 172.21.15.32:0/1072351304 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0ca7f0 con 0x7f7aec069e90 2023-10-16T08:17:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af1a2d640 1 --2- 172.21.15.32:0/1072351304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec069e90 0x7f7aec06d010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097c0 tx=0x7f7ae801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbaef8314998e71c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0c3a20 con 0x7f7aec069e90 2023-10-16T08:17:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae80ab650 con 0x7f7aec069e90 2023-10-16T08:17:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af0a2b640 1 -- 172.21.15.32:0/1072351304 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80ab9e0 con 0x7f7aec069e90 2023-10-16T08:17:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 -- 172.21.15.32:0/1072351304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec069e90 msgr2=0x7f7aec06d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 --2- 172.21.15.32:0/1072351304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec069e90 0x7f7aec06d010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097c0 tx=0x7f7ae801c450 comp rx=0 tx=0).stop 2023-10-16T08:17:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 -- 172.21.15.32:0/1072351304 shutdown_connections 2023-10-16T08:17:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 --2- 172.21.15.32:0/1072351304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec069e90 0x7f7aec06d010 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 -- 172.21.15.32:0/1072351304 >> 172.21.15.32:0/1072351304 conn(0x7f7aec0b0da0 msgr2=0x7f7aec0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 -- 172.21.15.32:0/1072351304 shutdown_connections 2023-10-16T08:17:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.725+0000 7f7af3c91640 1 -- 172.21.15.32:0/1072351304 wait complete. 2023-10-16T08:17:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 Processor -- start 2023-10-16T08:17:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- start start 2023-10-16T08:17:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aec069e90 0x7f7aec0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec06a270 0x7f7aec0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 0x7f7aec15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec15c620 con 0x7f7aec0c26b0 2023-10-16T08:17:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0ccf70 con 0x7f7aec06a270 2023-10-16T08:17:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0cd180 con 0x7f7aec069e90 2023-10-16T08:17:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 0x7f7aec15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 0x7f7aec15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59478/0 (socket says 172.21.15.32:59478) 2023-10-16T08:17:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 -- 172.21.15.32:0/2263390611 learned_addr learned my addr 172.21.15.32:0/2263390611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af1a2d640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aec069e90 0x7f7aec0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af122c640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec06a270 0x7f7aec0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 -- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aec069e90 msgr2=0x7f7aec0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aec069e90 0x7f7aec0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 -- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec06a270 msgr2=0x7f7aec0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec06a270 0x7f7aec0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 -- 172.21.15.32:0/2263390611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8009420 con 0x7f7aec0c26b0 2023-10-16T08:17:37.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af222e640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 0x7f7aec15b9d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c730 tx=0x7f7ae400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003b80 con 0x7f7aec0c26b0 2023-10-16T08:17:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4003ce0 con 0x7f7aec0c26b0 2023-10-16T08:17:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec0cd460 con 0x7f7aec0c26b0 2023-10-16T08:17:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400aa90 con 0x7f7aec0c26b0 2023-10-16T08:17:37.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec0cd7f0 con 0x7f7aec0c26b0 2023-10-16T08:17:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.729+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec0cdc20 con 0x7f7aec0c26b0 2023-10-16T08:17:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.733+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7ae400abf0 con 0x7f7aec0c26b0 2023-10-16T08:17:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.733+0000 7f7ae27fc640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ad00664a0 0x7f7ad0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:37.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.733+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ae4084060 con 0x7f7aec0c26b0 2023-10-16T08:17:37.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.733+0000 7f7af1a2d640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ad00664a0 0x7f7ad0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:37.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.733+0000 7f7af1a2d640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ad00664a0 0x7f7ad0068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80af000 tx=0x7f7ae80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:37.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.737+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7ae40b5090 con 0x7f7aec0c26b0 2023-10-16T08:17:37.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7aec04e170 con 0x7f7aec0c26b0 2023-10-16T08:17:37.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7ae27fc640 1 -- 172.21.15.32:0/2263390611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ae4051080 con 0x7f7aec0c26b0 2023-10-16T08:17:37.900 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:17:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ad00664a0 msgr2=0x7f7ad0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ad00664a0 0x7f7ad0068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80af000 tx=0x7f7ae80a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 msgr2=0x7f7aec15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 0x7f7aec15b9d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c730 tx=0x7f7ae400cbf0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 shutdown_connections 2023-10-16T08:17:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aec0c26b0 0x7f7aec15b9d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7aec06a270 0x7f7aec0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7ad00664a0 0x7f7ad0068960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 --2- 172.21.15.32:0/2263390611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7aec069e90 0x7f7aec0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 >> 172.21.15.32:0/2263390611 conn(0x7f7aec0b0da0 msgr2=0x7f7aec15a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 shutdown_connections 2023-10-16T08:17:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:37.893+0000 7f7af3c91640 1 -- 172.21.15.32:0/2263390611 wait complete. 2023-10-16T08:17:37.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:17:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 Processor -- start 2023-10-16T08:17:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 -- start start 2023-10-16T08:17:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb0e015cb90 0x7fb0e00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0e00d15d0 0x7fb0e00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00ce770 con 0x7fb0e00d3010 2023-10-16T08:17:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00d0600 con 0x7fb0e00d15d0 2023-10-16T08:17:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e6291640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00c6450 con 0x7fb0e015cb90 2023-10-16T08:17:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0e00d15d0 0x7fb0e00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0e00d15d0 0x7fb0e00c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59484/0 (socket says 172.21.15.32:59484) 2023-10-16T08:17:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 -- 172.21.15.32:0/378574305 learned_addr learned my addr 172.21.15.32:0/378574305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0dffff640 1 --2- 172.21.15.32:0/378574305 >> v2:172.21.15.186:3300/0 conn(0x7fb0e015cb90 0x7fb0e00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 -- 172.21.15.32:0/378574305 >> v2:172.21.15.186:3300/0 conn(0x7fb0e015cb90 msgr2=0x7fb0e00cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 --2- 172.21.15.32:0/378574305 >> v2:172.21.15.186:3300/0 conn(0x7fb0e015cb90 0x7fb0e00cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 -- 172.21.15.32:0/378574305 >> v1:172.21.15.186:6789/0 conn(0x7fb0e00d3010 legacy=0x7fb0e00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.057+0000 7fb0e482e640 1 -- 172.21.15.32:0/378574305 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00d08a0 con 0x7fb0e00d15d0 2023-10-16T08:17:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e482e640 1 --2- 172.21.15.32:0/378574305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d15d0 0x7fb0e00c6c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb0e008ba70 tx=0x7fb0d801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd205ae8da2070e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0deffd640 1 -- 172.21.15.32:0/378574305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a57d0 con 0x7fb0e00d15d0 2023-10-16T08:17:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0deffd640 1 -- 172.21.15.32:0/378574305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80a5930 con 0x7fb0e00d15d0 2023-10-16T08:17:38.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0deffd640 1 -- 172.21.15.32:0/378574305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a5cc0 con 0x7fb0e00d15d0 2023-10-16T08:17:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 -- 172.21.15.32:0/378574305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d15d0 msgr2=0x7fb0e00c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 --2- 172.21.15.32:0/378574305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d15d0 0x7fb0e00c6c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb0e008ba70 tx=0x7fb0d801c910 comp rx=0 tx=0).stop 2023-10-16T08:17:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 -- 172.21.15.32:0/378574305 shutdown_connections 2023-10-16T08:17:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 --2- 172.21.15.32:0/378574305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d15d0 0x7fb0e00c6c10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 --2- 172.21.15.32:0/378574305 >> v2:172.21.15.186:3300/0 conn(0x7fb0e015cb90 0x7fb0e00cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 -- 172.21.15.32:0/378574305 >> 172.21.15.32:0/378574305 conn(0x7fb0e009bc50 msgr2=0x7fb0e00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 -- 172.21.15.32:0/378574305 shutdown_connections 2023-10-16T08:17:38.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.061+0000 7fb0e6291640 1 -- 172.21.15.32:0/378574305 wait complete. 2023-10-16T08:17:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 Processor -- start 2023-10-16T08:17:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 -- start start 2023-10-16T08:17:38.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e00d15d0 0x7fb0e014d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e00d3010 0x7fb0e014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 0x7fb0e015b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0159b70 con 0x7fb0e015cb90 2023-10-16T08:17:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0153b30 con 0x7fb0e00d3010 2023-10-16T08:17:38.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e01540f0 con 0x7fb0e00d15d0 2023-10-16T08:17:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 0x7fb0e015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 0x7fb0e015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59500/0 (socket says 172.21.15.32:59500) 2023-10-16T08:17:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 -- 172.21.15.32:0/2513960003 learned_addr learned my addr 172.21.15.32:0/2513960003 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0dffff640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e00d15d0 0x7fb0e014d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0df7fe640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e00d3010 0x7fb0e014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 -- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e00d15d0 msgr2=0x7fb0e014d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e00d15d0 0x7fb0e014d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 -- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e00d3010 msgr2=0x7fb0e014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e00d3010 0x7fb0e014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 -- 172.21.15.32:0/2513960003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8009b30 con 0x7fb0e015cb90 2023-10-16T08:17:38.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e482e640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 0x7fb0e015b210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8019520 tx=0x7fb0d80196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b8670 con 0x7fb0e015cb90 2023-10-16T08:17:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80b87d0 con 0x7fb0e015cb90 2023-10-16T08:17:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b8b80 con 0x7fb0e015cb90 2023-10-16T08:17:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00cdc80 con 0x7fb0e015cb90 2023-10-16T08:17:38.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.065+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e0153d80 con 0x7fb0e015cb90 2023-10-16T08:17:38.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.069+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0154260 con 0x7fb0e015cb90 2023-10-16T08:17:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.069+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb0d80b8ce0 con 0x7fb0e015cb90 2023-10-16T08:17:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.069+0000 7fb0dcff9640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb0cc0664f0 0x7fb0cc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.069+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb0d811f780 con 0x7fb0e015cb90 2023-10-16T08:17:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.069+0000 7fb0dffff640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb0cc0664f0 0x7fb0cc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.069+0000 7fb0dffff640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb0cc0664f0 0x7fb0cc0689b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00079a0 tx=0x7fb0d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.073+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0d80ec960 con 0x7fb0e015cb90 2023-10-16T08:17:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.261+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0e0074010 con 0x7fb0e015cb90 2023-10-16T08:17:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.261+0000 7fb0dcff9640 1 -- 172.21.15.32:0/2513960003 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7fb0d80eff80 con 0x7fb0e015cb90 2023-10-16T08:17:38.270 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:38.270 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:17:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:17:38.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb0cc0664f0 msgr2=0x7fb0cc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb0cc0664f0 0x7fb0cc0689b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00079a0 tx=0x7fb0d0008040 comp rx=0 tx=0).stop 2023-10-16T08:17:38.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 msgr2=0x7fb0e015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 0x7fb0e015b210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8019520 tx=0x7fb0d80196b0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 shutdown_connections 2023-10-16T08:17:38.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e015cb90 0x7fb0e015b210 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e00d3010 0x7fb0e014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb0cc0664f0 0x7fb0cc0689b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 --2- 172.21.15.32:0/2513960003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb0e00d15d0 0x7fb0e014d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 >> 172.21.15.32:0/2513960003 conn(0x7fb0e009bc50 msgr2=0x7fb0e00c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:38.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 shutdown_connections 2023-10-16T08:17:38.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.265+0000 7fb0e6291640 1 -- 172.21.15.32:0/2513960003 wait complete. 2023-10-16T08:17:38.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 Processor -- start 2023-10-16T08:17:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 -- start start 2023-10-16T08:17:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2b9c0b2ad0 0x7f2b9c06b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b9c0ce310 0x7f2b9c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c06d000 con 0x7f2b9c15cf90 2023-10-16T08:17:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0ced00 con 0x7f2b9c0ce310 2023-10-16T08:17:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2ba0f94640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c160580 con 0x7f2b9c0b2ad0 2023-10-16T08:17:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b9c0ce310 0x7f2b9c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b9c0ce310 0x7f2b9c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59512/0 (socket says 172.21.15.32:59512) 2023-10-16T08:17:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/1591869856 learned_addr learned my addr 172.21.15.32:0/1591869856 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:38.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9a59c640 1 --2- 172.21.15.32:0/1591869856 >> v2:172.21.15.186:3300/0 conn(0x7f2b9c0b2ad0 0x7f2b9c06b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/1591869856 >> v2:172.21.15.186:3300/0 conn(0x7f2b9c0b2ad0 msgr2=0x7f2b9c06b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:38.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 --2- 172.21.15.32:0/1591869856 >> v2:172.21.15.186:3300/0 conn(0x7f2b9c0b2ad0 0x7f2b9c06b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/1591869856 >> v1:172.21.15.186:6789/0 conn(0x7f2b9c15cf90 legacy=0x7f2b9c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.429+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/1591869856 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c15d4e0 con 0x7f2b9c0ce310 2023-10-16T08:17:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2b9ad9d640 1 --2- 172.21.15.32:0/1591869856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c0ce310 0x7f2b9c0ce7c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009e50 tx=0x7f2b9401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d46bf8e6a7a0cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2b9959a640 1 -- 172.21.15.32:0/1591869856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b940a5860 con 0x7f2b9c0ce310 2023-10-16T08:17:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2b9959a640 1 -- 172.21.15.32:0/1591869856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b940a59c0 con 0x7f2b9c0ce310 2023-10-16T08:17:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2b9959a640 1 -- 172.21.15.32:0/1591869856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b940a5d50 con 0x7f2b9c0ce310 2023-10-16T08:17:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 -- 172.21.15.32:0/1591869856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c0ce310 msgr2=0x7f2b9c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/1591869856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c0ce310 0x7f2b9c0ce7c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009e50 tx=0x7f2b9401c9a0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 -- 172.21.15.32:0/1591869856 shutdown_connections 2023-10-16T08:17:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/1591869856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c0ce310 0x7f2b9c0ce7c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/1591869856 >> v2:172.21.15.186:3300/0 conn(0x7f2b9c0b2ad0 0x7f2b9c06b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 -- 172.21.15.32:0/1591869856 >> 172.21.15.32:0/1591869856 conn(0x7f2b9c0cd1c0 msgr2=0x7f2b9c0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 -- 172.21.15.32:0/1591869856 shutdown_connections 2023-10-16T08:17:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.433+0000 7f2ba0f94640 1 -- 172.21.15.32:0/1591869856 wait complete. 2023-10-16T08:17:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 Processor -- start 2023-10-16T08:17:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 -- start start 2023-10-16T08:17:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c0b2ad0 0x7f2b9c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c0ce310 0x7f2b9c0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 0x7f2b9c0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b4bd0 con 0x7f2b9c15cf90 2023-10-16T08:17:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b4eb0 con 0x7f2b9c0b2ad0 2023-10-16T08:17:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0c43f0 con 0x7f2b9c0ce310 2023-10-16T08:17:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 0x7f2b9c0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 0x7f2b9c0c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59526/0 (socket says 172.21.15.32:59526) 2023-10-16T08:17:38.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/3987805500 learned_addr learned my addr 172.21.15.32:0/3987805500 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:38.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9a59c640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c0b2ad0 0x7f2b9c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b99d9b640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c0ce310 0x7f2b9c0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c0ce310 msgr2=0x7f2b9c0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c0ce310 0x7f2b9c0b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c0b2ad0 msgr2=0x7f2b9c0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c0b2ad0 0x7f2b9c0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 -- 172.21.15.32:0/3987805500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009b30 con 0x7f2b9c15cf90 2023-10-16T08:17:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b9ad9d640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 0x7f2b9c0c5e40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b940195b0 tx=0x7f2b94019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b940b8670 con 0x7f2b9c15cf90 2023-10-16T08:17:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b940b87d0 con 0x7f2b9c15cf90 2023-10-16T08:17:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c0c46d0 con 0x7f2b9c15cf90 2023-10-16T08:17:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b940b8b60 con 0x7f2b9c15cf90 2023-10-16T08:17:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.437+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c0b47f0 con 0x7f2b9c15cf90 2023-10-16T08:17:38.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.441+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c0c4250 con 0x7f2b9c15cf90 2023-10-16T08:17:38.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.441+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2b940b8cc0 con 0x7f2b9c15cf90 2023-10-16T08:17:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.441+0000 7f2b8affd640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2b7c0664a0 0x7f2b7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.441+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b9411f730 con 0x7f2b9c15cf90 2023-10-16T08:17:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.441+0000 7f2b9a59c640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2b7c0664a0 0x7f2b7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.441+0000 7f2b9a59c640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2b7c0664a0 0x7f2b7c068960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009ac0 tx=0x7f2b8c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.445+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2b9c0c4250 con 0x7f2b9c15cf90 2023-10-16T08:17:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.601+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b9c0c4910 con 0x7f2b9c15cf90 2023-10-16T08:17:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.601+0000 7f2b8affd640 1 -- 172.21.15.32:0/3987805500 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7f2b9c0c4910 con 0x7f2b9c15cf90 2023-10-16T08:17:38.609 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:38.610 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:38.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2b7c0664a0 msgr2=0x7f2b7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2b7c0664a0 0x7f2b7c068960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009ac0 tx=0x7f2b8c009190 comp rx=0 tx=0).stop 2023-10-16T08:17:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 msgr2=0x7f2b9c0c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 0x7f2b9c0c5e40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b940195b0 tx=0x7f2b94019740 comp rx=0 tx=0).stop 2023-10-16T08:17:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 shutdown_connections 2023-10-16T08:17:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9c15cf90 0x7f2b9c0c5e40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2b7c0664a0 0x7f2b7c068960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2b9c0ce310 0x7f2b9c0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 --2- 172.21.15.32:0/3987805500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b9c0b2ad0 0x7f2b9c0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 >> 172.21.15.32:0/3987805500 conn(0x7f2b9c0cd1c0 msgr2=0x7f2b9c15ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 shutdown_connections 2023-10-16T08:17:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.605+0000 7f2ba0f94640 1 -- 172.21.15.32:0/3987805500 wait complete. 2023-10-16T08:17:38.631 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T08:17:38.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:17:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 Processor -- start 2023-10-16T08:17:38.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 -- start start 2023-10-16T08:17:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41381597c0 0x7f41380ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41380723d0 con 0x7f4138159ba0 2023-10-16T08:17:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41380717c0 con 0x7f41381597c0 2023-10-16T08:17:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f413e51d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4138071aa0 con 0x7f41380adc10 2023-10-16T08:17:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f4137fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41381597c0 0x7f41380ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f4137fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41381597c0 0x7f41380ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59534/0 (socket says 172.21.15.32:59534) 2023-10-16T08:17:38.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.769+0000 7f4137fff640 1 -- 172.21.15.32:0/1149137201 learned_addr learned my addr 172.21.15.32:0/1149137201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:38.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f413caba640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4137fff640 1 -- 172.21.15.32:0/1149137201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 msgr2=0x7f41380cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4137fff640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4137fff640 1 -- 172.21.15.32:0/1149137201 >> v1:172.21.15.186:6789/0 conn(0x7f4138159ba0 legacy=0x7f41380cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4137fff640 1 -- 172.21.15.32:0/1149137201 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4138071d80 con 0x7f41381597c0 2023-10-16T08:17:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f413caba640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4137fff640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f41380ad6d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420495bd6df449f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4136ffd640 1 -- 172.21.15.32:0/1149137201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280aa640 con 0x7f41381597c0 2023-10-16T08:17:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4136ffd640 1 -- 172.21.15.32:0/1149137201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41280aa7a0 con 0x7f41381597c0 2023-10-16T08:17:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.773+0000 7f4136ffd640 1 -- 172.21.15.32:0/1149137201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280aab30 con 0x7f41381597c0 2023-10-16T08:17:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- 172.21.15.32:0/1149137201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 msgr2=0x7f41380ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f41380ad6d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412801c540 comp rx=0 tx=0).stop 2023-10-16T08:17:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- 172.21.15.32:0/1149137201 shutdown_connections 2023-10-16T08:17:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 --2- 172.21.15.32:0/1149137201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f41380ad6d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- 172.21.15.32:0/1149137201 >> 172.21.15.32:0/1149137201 conn(0x7f413806c190 msgr2=0x7f413806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- 172.21.15.32:0/1149137201 shutdown_connections 2023-10-16T08:17:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- 172.21.15.32:0/1149137201 wait complete. 2023-10-16T08:17:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 Processor -- start 2023-10-16T08:17:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- start start 2023-10-16T08:17:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f413814e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4138159ba0 0x7f413814eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413814c370 con 0x7f41381597c0 2023-10-16T08:17:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380c6cc0 con 0x7f4138159ba0 2023-10-16T08:17:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413e51d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41381f9850 con 0x7f41380adc10 2023-10-16T08:17:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f41377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f413814e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f41377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f413814e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59542/0 (socket says 172.21.15.32:59542) 2023-10-16T08:17:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f41377fe640 1 -- 172.21.15.32:0/3738957363 learned_addr learned my addr 172.21.15.32:0/3738957363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f4137fff640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.777+0000 7f413caba640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4138159ba0 0x7f413814eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f41377fe640 1 -- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 msgr2=0x7f41380b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f41377fe640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f41377fe640 1 -- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4138159ba0 msgr2=0x7f413814eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f41377fe640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4138159ba0 0x7f413814eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f41377fe640 1 -- 172.21.15.32:0/3738957363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280094d0 con 0x7f41381597c0 2023-10-16T08:17:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f413caba640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4138159ba0 0x7f413814eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f41377fe640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f413814e760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f412c00c420 tx=0x7f412c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0052d0 con 0x7f41381597c0 2023-10-16T08:17:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c005430 con 0x7f41381597c0 2023-10-16T08:17:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c003840 con 0x7f41381597c0 2023-10-16T08:17:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41380b3940 con 0x7f41381597c0 2023-10-16T08:17:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41380b3f00 con 0x7f41381597c0 2023-10-16T08:17:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41380c6e50 con 0x7f41381597c0 2023-10-16T08:17:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f412c0039a0 con 0x7f41381597c0 2023-10-16T08:17:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f4134ff9640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41240664a0 0x7f4124068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.781+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f412c083730 con 0x7f41381597c0 2023-10-16T08:17:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.785+0000 7f4137fff640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41240664a0 0x7f4124068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.785+0000 7f4137fff640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41240664a0 0x7f4124068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f41280ae000 tx=0x7f41280a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.785+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f412c050750 con 0x7f41381597c0 2023-10-16T08:17:38.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.973+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41380cd3f0 con 0x7f41381597c0 2023-10-16T08:17:38.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.973+0000 7f4134ff9640 1 -- 172.21.15.32:0/3738957363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f412c053d70 con 0x7f41381597c0 2023-10-16T08:17:38.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:38.981 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:17:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:17:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41240664a0 msgr2=0x7f4124068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41240664a0 0x7f4124068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f41280ae000 tx=0x7f41280a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 msgr2=0x7f413814e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f413814e760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f412c00c420 tx=0x7f412c00c8e0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 shutdown_connections 2023-10-16T08:17:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4138159ba0 0x7f413814eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41381597c0 0x7f413814e760 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41240664a0 0x7f4124068960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 --2- 172.21.15.32:0/3738957363 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41380adc10 0x7f41380b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 >> 172.21.15.32:0/3738957363 conn(0x7f413806c190 msgr2=0x7f41380af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 shutdown_connections 2023-10-16T08:17:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:38.977+0000 7f413e51d640 1 -- 172.21.15.32:0/3738957363 wait complete. 2023-10-16T08:17:39.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:17:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764e33e640 1 Processor -- start 2023-10-16T08:17:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764e33e640 1 -- start start 2023-10-16T08:17:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764e33e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f76480d3010 0x7f76480cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764e33e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76480ce770 con 0x7f764815cb90 2023-10-16T08:17:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764e33e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76480d0600 con 0x7f76480d15d0 2023-10-16T08:17:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764e33e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76480c6450 con 0x7f76480d3010 2023-10-16T08:17:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7647fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f764815cb90 0x7f76480d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42252/0 (socket says 172.21.15.32:42252) 2023-10-16T08:17:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7647fff640 1 -- 172.21.15.32:0/3899000058 learned_addr learned my addr 172.21.15.32:0/3899000058 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428693191 0 0) 0x7f76480ce770 con 0x7f764815cb90 2023-10-16T08:17:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764c8db640 1 --2- 172.21.15.32:0/3899000058 >> v2:172.21.15.186:3300/0 conn(0x7f76480d3010 0x7f76480cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7630003530 con 0x7f764815cb90 2023-10-16T08:17:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114408024 0 0) 0x7f7630003530 con 0x7f764815cb90 2023-10-16T08:17:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 >> v2:172.21.15.186:3300/0 conn(0x7f76480d3010 msgr2=0x7f76480cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 --2- 172.21.15.32:0/3899000058 >> v2:172.21.15.186:3300/0 conn(0x7f76480d3010 0x7f76480cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 >> v1:172.21.15.186:6789/0 conn(0x7f76480d15d0 legacy=0x7f76480c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480d08a0 con 0x7f764815cb90 2023-10-16T08:17:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f7638001800 con 0x7f764815cb90 2023-10-16T08:17:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 >> v1:172.21.15.32:6789/0 conn(0x7f764815cb90 legacy=0x7f76480d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7630003980 0x7f7630003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76300047d0 0x7f7630004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630005190 0x7f763000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76480c6450 con 0x7f76300047d0 2023-10-16T08:17:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763000c130 con 0x7f7630005190 2023-10-16T08:17:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763000c2f0 con 0x7f7630003980 2023-10-16T08:17:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f76477fe640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76300047d0 0x7f7630004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f764c8db640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630005190 0x7f763000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.145+0000 7f7647fff640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7630003980 0x7f7630003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f76477fe640 1 -- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7630003980 msgr2=0x7f7630003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f76477fe640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7630003980 0x7f7630003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f76477fe640 1 -- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630005190 msgr2=0x7f763000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f76477fe640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7630005190 0x7f763000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f76477fe640 1 -- 172.21.15.32:0/3899000058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630003530 con 0x7f76300047d0 2023-10-16T08:17:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f764c8db640 1 -- 172.21.15.32:0/3899000058 reap_dead start 2023-10-16T08:17:39.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f76477fe640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76300047d0 0x7f7630004c50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f763c001780 tx=0x7f763c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb7fa7b285bf849 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0052d0 con 0x7f76300047d0 2023-10-16T08:17:39.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0a7050 con 0x7f76300047d0 2023-10-16T08:17:39.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f7646ffd640 1 -- 172.21.15.32:0/3899000058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c003600 con 0x7f76300047d0 2023-10-16T08:17:39.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f764e33e640 1 -- 172.21.15.32:0/3899000058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76300047d0 msgr2=0x7f7630004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f764e33e640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76300047d0 0x7f7630004c50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f763c001780 tx=0x7f763c01c3b0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f764e33e640 1 -- 172.21.15.32:0/3899000058 shutdown_connections 2023-10-16T08:17:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f764e33e640 1 --2- 172.21.15.32:0/3899000058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76300047d0 0x7f7630004c50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.149+0000 7f764e33e640 1 -- 172.21.15.32:0/3899000058 >> 172.21.15.32:0/3899000058 conn(0x7f764809bc50 msgr2=0x7f76480d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 -- 172.21.15.32:0/3899000058 shutdown_connections 2023-10-16T08:17:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 -- 172.21.15.32:0/3899000058 wait complete. 2023-10-16T08:17:39.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 Processor -- start 2023-10-16T08:17:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 -- start start 2023-10-16T08:17:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76300047d0 0x7f76481e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764814ad10 0x7f76480cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 0x7f76480cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76480b0c80 con 0x7f76480cb540 2023-10-16T08:17:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76480b2e80 con 0x7f764814ad10 2023-10-16T08:17:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764e33e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76480b0630 con 0x7f76300047d0 2023-10-16T08:17:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 0x7f76480cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 0x7f76480cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59562/0 (socket says 172.21.15.32:59562) 2023-10-16T08:17:39.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 -- 172.21.15.32:0/2091344318 learned_addr learned my addr 172.21.15.32:0/2091344318 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f7647fff640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76300047d0 0x7f76481e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f76477fe640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764814ad10 0x7f76480cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 -- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76300047d0 msgr2=0x7f76481e84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76300047d0 0x7f76481e84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 -- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764814ad10 msgr2=0x7f76480cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764814ad10 0x7f76480cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f764c8db640 1 -- 172.21.15.32:0/2091344318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480b30d0 con 0x7f76480cb540 2023-10-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.153+0000 7f7647fff640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76300047d0 0x7f76481e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f764c8db640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 0x7f76480cb9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7640011bf0 tx=0x7f764000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000e040 con 0x7f76480cb540 2023-10-16T08:17:39.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000f040 con 0x7f76480cb540 2023-10-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640015430 con 0x7f76480cb540 2023-10-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76480b08a0 con 0x7f76480cb540 2023-10-16T08:17:39.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764814b190 con 0x7f76480cb540 2023-10-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7648074010 con 0x7f76480cb540 2023-10-16T08:17:39.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7640015590 con 0x7f76480cb540 2023-10-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7644ff9640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7640084c80 con 0x7f76480cb540 2023-10-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.157+0000 7f7647fff640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.161+0000 7f7647fff640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7638000f00 tx=0x7f7638009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.161+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7640051d30 con 0x7f76480cb540 2023-10-16T08:17:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.353+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76480c7fc0 con 0x7f76480cb540 2023-10-16T08:17:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.353+0000 7f7644ff9640 1 -- 172.21.15.32:0/2091344318 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4909 (secure 0 0 0) 0x7f7640084070 con 0x7f76480cb540 2023-10-16T08:17:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:39.362 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/3746896482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3746896482},{"type":"v1","addr":"172.21.15.186:6837","nonce":3746896482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2239441787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":2239441787},{"type":"v1","addr":"172.21.15.186:6836","nonce":2239441787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/18224204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":18224204},{"type":"v1","addr":"172.21.15.32:6833","nonce":18224204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:16:22.323436+0000","modified":"2023-10-16T08:16:46.160332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4311":{"gid":4311,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1428800105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1428800105},{"type":"v1","addr":"172.21.15.32:6835","nonce":1428800105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:17:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-10-16T08:17:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 msgr2=0x7f7634068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7638000f00 tx=0x7f7638009040 comp rx=0 tx=0).stop 2023-10-16T08:17:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 msgr2=0x7f76480cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 0x7f76480cb9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7640011bf0 tx=0x7f764000f580 comp rx=0 tx=0).stop 2023-10-16T08:17:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 shutdown_connections 2023-10-16T08:17:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76480cb540 0x7f76480cb9c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764814ad10 0x7f76480cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 --2- 172.21.15.32:0/2091344318 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76300047d0 0x7f76481e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 >> 172.21.15.32:0/2091344318 conn(0x7f764809bc50 msgr2=0x7f76480b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 shutdown_connections 2023-10-16T08:17:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.357+0000 7f764e33e640 1 -- 172.21.15.32:0/2091344318 wait complete. 2023-10-16T08:17:39.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 Processor -- start 2023-10-16T08:17:39.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 -- start start 2023-10-16T08:17:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdabc160320 0x7fdabc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0b16f0 0x7fdabc0d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0d30b0 con 0x7fdabc0ce5e0 2023-10-16T08:17:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0d3390 con 0x7fdabc160320 2023-10-16T08:17:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac3972640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0d3670 con 0x7fdabc0b16f0 2023-10-16T08:17:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdabc160320 0x7fdabc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdabc160320 0x7fdabc0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59576/0 (socket says 172.21.15.32:59576) 2023-10-16T08:17:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 -- 172.21.15.32:0/3741867917 learned_addr learned my addr 172.21.15.32:0/3741867917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac1f0f640 1 --2- 172.21.15.32:0/3741867917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0b16f0 0x7fdabc0d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 -- 172.21.15.32:0/3741867917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0b16f0 msgr2=0x7fdabc0d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 --2- 172.21.15.32:0/3741867917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0b16f0 0x7fdabc0d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 -- 172.21.15.32:0/3741867917 >> v1:172.21.15.186:6789/0 conn(0x7fdabc0ce5e0 legacy=0x7fdabc0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.521+0000 7fdac170e640 1 -- 172.21.15.32:0/3741867917 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0ce0c0 con 0x7fdabc160320 2023-10-16T08:17:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac170e640 1 --2- 172.21.15.32:0/3741867917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc160320 0x7fdabc0cf1d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdab8006fd0 tx=0x7fdab801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328d997d127a0c49 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdab3fff640 1 -- 172.21.15.32:0/3741867917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80ab600 con 0x7fdabc160320 2023-10-16T08:17:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdab3fff640 1 -- 172.21.15.32:0/3741867917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab80a7050 con 0x7fdabc160320 2023-10-16T08:17:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdab3fff640 1 -- 172.21.15.32:0/3741867917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80ab980 con 0x7fdabc160320 2023-10-16T08:17:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 -- 172.21.15.32:0/3741867917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc160320 msgr2=0x7fdabc0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 --2- 172.21.15.32:0/3741867917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc160320 0x7fdabc0cf1d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdab8006fd0 tx=0x7fdab801c490 comp rx=0 tx=0).stop 2023-10-16T08:17:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 -- 172.21.15.32:0/3741867917 shutdown_connections 2023-10-16T08:17:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 --2- 172.21.15.32:0/3741867917 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc0b16f0 0x7fdabc0d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 --2- 172.21.15.32:0/3741867917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc160320 0x7fdabc0cf1d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 -- 172.21.15.32:0/3741867917 >> 172.21.15.32:0/3741867917 conn(0x7fdabc09bc50 msgr2=0x7fdabc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 -- 172.21.15.32:0/3741867917 shutdown_connections 2023-10-16T08:17:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 -- 172.21.15.32:0/3741867917 wait complete. 2023-10-16T08:17:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.525+0000 7fdac3972640 1 Processor -- start 2023-10-16T08:17:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- start start 2023-10-16T08:17:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 0x7fdabc0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0ce5e0 0x7fdabc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc160320 0x7fdabc14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0ca340 con 0x7fdabc0b16f0 2023-10-16T08:17:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0cc750 con 0x7fdabc0ce5e0 2023-10-16T08:17:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc14c740 con 0x7fdabc160320 2023-10-16T08:17:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 0x7fdabc0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 0x7fdabc0c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59586/0 (socket says 172.21.15.32:59586) 2023-10-16T08:17:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 -- 172.21.15.32:0/2235269569 learned_addr learned my addr 172.21.15.32:0/2235269569 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac1f0f640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc160320 0x7fdabc14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac0f0d640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0ce5e0 0x7fdabc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 -- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc160320 msgr2=0x7fdabc14c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc160320 0x7fdabc14c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 -- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0ce5e0 msgr2=0x7fdabc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0ce5e0 0x7fdabc0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 -- 172.21.15.32:0/2235269569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009480 con 0x7fdabc0b16f0 2023-10-16T08:17:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac1f0f640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc160320 0x7fdabc14c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:39.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac0f0d640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0ce5e0 0x7fdabc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:39.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac170e640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 0x7fdabc0c9e00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdab801c970 tx=0x7fdab8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8003e60 con 0x7fdabc0b16f0 2023-10-16T08:17:39.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab80059a0 con 0x7fdabc0b16f0 2023-10-16T08:17:39.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc0ca530 con 0x7fdabc0b16f0 2023-10-16T08:17:39.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8005d50 con 0x7fdabc0b16f0 2023-10-16T08:17:39.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc0caaf0 con 0x7fdabc0b16f0 2023-10-16T08:17:39.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.529+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc074010 con 0x7fdabc0b16f0 2023-10-16T08:17:39.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.533+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdab80b2040 con 0x7fdabc0b16f0 2023-10-16T08:17:39.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.533+0000 7fdab1ffb640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdaa00664d0 0x7fdaa0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.533+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdab811dcd0 con 0x7fdabc0b16f0 2023-10-16T08:17:39.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.533+0000 7fdac0f0d640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdaa00664d0 0x7fdaa0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.533+0000 7fdac0f0d640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdaa00664d0 0x7fdaa0068990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdabc1f8ca0 tx=0x7fdaac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.537+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdab80ead80 con 0x7fdabc0b16f0 2023-10-16T08:17:39.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdabc06ccf0 con 0x7fdabc0b16f0 2023-10-16T08:17:39.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdab1ffb640 1 -- 172.21.15.32:0/2235269569 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18734 (secure 0 0 0) 0x7fdab80b0090 con 0x7fdabc0b16f0 2023-10-16T08:17:39.704 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:39.704 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:03.851769+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:16:20.036424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:16:21.327248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T08:16:25.022345+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:39.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdaa00664d0 msgr2=0x7fdaa0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdaa00664d0 0x7fdaa0068990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdabc1f8ca0 tx=0x7fdaac009040 comp rx=0 tx=0).stop 2023-10-16T08:17:39.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 msgr2=0x7fdabc0c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 0x7fdabc0c9e00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdab801c970 tx=0x7fdab8019230 comp rx=0 tx=0).stop 2023-10-16T08:17:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 shutdown_connections 2023-10-16T08:17:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdaa00664d0 0x7fdaa0068990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdabc160320 0x7fdabc14c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdabc0ce5e0 0x7fdabc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 --2- 172.21.15.32:0/2235269569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdabc0b16f0 0x7fdabc0c9e00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.697+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 >> 172.21.15.32:0/2235269569 conn(0x7fdabc09bc50 msgr2=0x7fdabc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:39.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.701+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 shutdown_connections 2023-10-16T08:17:39.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.701+0000 7fdac3972640 1 -- 172.21.15.32:0/2235269569 wait complete. 2023-10-16T08:17:39.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T08:17:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.861+0000 7fe1535f8640 1 Processor -- start 2023-10-16T08:17:39.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.861+0000 7fe1535f8640 1 -- start start 2023-10-16T08:17:39.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe1535f8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe14c15ca70 0x7fe14c0d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe1535f8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe14c0d3550 0x7fe14c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe1535f8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c0cfa00 con 0x7fe14c0cf5d0 2023-10-16T08:17:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe1535f8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c0ce820 con 0x7fe14c15ca70 2023-10-16T08:17:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe1535f8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c0c65f0 con 0x7fe14c0d3550 2023-10-16T08:17:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe14c15ca70 0x7fe14c0d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151b95640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe14c0cf5d0 0x7fe14c0d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42266/0 (socket says 172.21.15.32:42266) 2023-10-16T08:17:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe14c15ca70 0x7fe14c0d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59594/0 (socket says 172.21.15.32:59594) 2023-10-16T08:17:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151b95640 1 -- 172.21.15.32:0/1046531105 learned_addr learned my addr 172.21.15.32:0/1046531105 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe150b93640 1 --2- 172.21.15.32:0/1046531105 >> v2:172.21.15.186:3300/0 conn(0x7fe14c0d3550 0x7fe14c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 -- 172.21.15.32:0/1046531105 >> v2:172.21.15.186:3300/0 conn(0x7fe14c0d3550 msgr2=0x7fe14c0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 --2- 172.21.15.32:0/1046531105 >> v2:172.21.15.186:3300/0 conn(0x7fe14c0d3550 0x7fe14c0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 -- 172.21.15.32:0/1046531105 >> v1:172.21.15.32:6789/0 conn(0x7fe14c0cf5d0 legacy=0x7fe14c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 -- 172.21.15.32:0/1046531105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0cdf80 con 0x7fe14c15ca70 2023-10-16T08:17:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe143fff640 1 -- 172.21.15.32:0/1046531105 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496020747 0 0) 0x7fe14c0cfa00 con 0x7fe14c0cf5d0 2023-10-16T08:17:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.865+0000 7fe151394640 1 --2- 172.21.15.32:0/1046531105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c15ca70 0x7fe14c0d3010 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0b54f0 tx=0x7fe14801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147ed378d2e062a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe143fff640 1 -- 172.21.15.32:0/1046531105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480ab610 con 0x7fe14c15ca70 2023-10-16T08:17:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe143fff640 1 -- 172.21.15.32:0/1046531105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480ab770 con 0x7fe14c15ca70 2023-10-16T08:17:39.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe143fff640 1 -- 172.21.15.32:0/1046531105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480abb00 con 0x7fe14c15ca70 2023-10-16T08:17:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- 172.21.15.32:0/1046531105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c15ca70 msgr2=0x7fe14c0d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 --2- 172.21.15.32:0/1046531105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c15ca70 0x7fe14c0d3010 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0b54f0 tx=0x7fe14801c410 comp rx=0 tx=0).stop 2023-10-16T08:17:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- 172.21.15.32:0/1046531105 shutdown_connections 2023-10-16T08:17:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 --2- 172.21.15.32:0/1046531105 >> v2:172.21.15.186:3300/0 conn(0x7fe14c0d3550 0x7fe14c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 --2- 172.21.15.32:0/1046531105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c15ca70 0x7fe14c0d3010 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- 172.21.15.32:0/1046531105 >> 172.21.15.32:0/1046531105 conn(0x7fe14c09bc50 msgr2=0x7fe14c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- 172.21.15.32:0/1046531105 shutdown_connections 2023-10-16T08:17:39.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- 172.21.15.32:0/1046531105 wait complete. 2023-10-16T08:17:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 Processor -- start 2023-10-16T08:17:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- start start 2023-10-16T08:17:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c0cf5d0 0x7fe14c0d2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 0x7fe14c0d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c15ca70 0x7fe14c155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c15bf90 con 0x7fe14c0d3550 2023-10-16T08:17:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c160090 con 0x7fe14c0cf5d0 2023-10-16T08:17:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe1535f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c9db0 con 0x7fe14c15ca70 2023-10-16T08:17:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe150b93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 0x7fe14c0d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe150b93640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 0x7fe14c0d2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59610/0 (socket says 172.21.15.32:59610) 2023-10-16T08:17:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.869+0000 7fe150b93640 1 -- 172.21.15.32:0/578614785 learned_addr learned my addr 172.21.15.32:0/578614785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe151b95640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c15ca70 0x7fe14c155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe151394640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c0cf5d0 0x7fe14c0d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe150b93640 1 -- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c15ca70 msgr2=0x7fe14c155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe150b93640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c15ca70 0x7fe14c155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe150b93640 1 -- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c0cf5d0 msgr2=0x7fe14c0d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:39.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe150b93640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c0cf5d0 0x7fe14c0d2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:39.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe150b93640 1 -- 172.21.15.32:0/578614785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480093b0 con 0x7fe14c0d3550 2023-10-16T08:17:39.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe151394640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c0cf5d0 0x7fe14c0d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:39.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe150b93640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 0x7fe14c0d2d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00d800 tx=0x7fe13c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00b6c0 con 0x7fe14c0d3550 2023-10-16T08:17:39.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c00b820 con 0x7fe14c0d3550 2023-10-16T08:17:39.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00bb90 con 0x7fe14c0d3550 2023-10-16T08:17:39.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c0ca090 con 0x7fe14c0d3550 2023-10-16T08:17:39.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c15fd20 con 0x7fe14c0d3550 2023-10-16T08:17:39.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c0b6920 con 0x7fe14c0d3550 2023-10-16T08:17:39.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.873+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe13c00bcf0 con 0x7fe14c0d3550 2023-10-16T08:17:39.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.877+0000 7fe141ffb640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1300664a0 0x7fe130068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:39.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.877+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe13c0847d0 con 0x7fe14c0d3550 2023-10-16T08:17:39.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.877+0000 7fe151394640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1300664a0 0x7fe130068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:39.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.877+0000 7fe151394640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1300664a0 0x7fe130068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe1480af000 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:39.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:39.877+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe13c051890 con 0x7fe14c0d3550 2023-10-16T08:17:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.069+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe14c15fee0 con 0x7fe14c0d3550 2023-10-16T08:17:40.078 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-10-16T08:17:40.073+0000 7efeb4e3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T08:17:40.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.073+0000 7fe141ffb640 1 -- 172.21.15.32:0/578614785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe13c011340 con 0x7fe14c0d3550 2023-10-16T08:17:40.081 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T08:17:40.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1300664a0 msgr2=0x7fe130068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1300664a0 0x7fe130068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe1480af000 tx=0x7fe148008040 comp rx=0 tx=0).stop 2023-10-16T08:17:40.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 msgr2=0x7fe14c0d2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 0x7fe14c0d2d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00d800 tx=0x7fe13c00dcc0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 shutdown_connections 2023-10-16T08:17:40.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1300664a0 0x7fe130068960 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe14c15ca70 0x7fe14c155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe14c0d3550 0x7fe14c0d2d50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 --2- 172.21.15.32:0/578614785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe14c0cf5d0 0x7fe14c0d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 >> 172.21.15.32:0/578614785 conn(0x7fe14c09bc50 msgr2=0x7fe14c0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:40.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 shutdown_connections 2023-10-16T08:17:40.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.077+0000 7fe1535f8640 1 -- 172.21.15.32:0/578614785 wait complete. 2023-10-16T08:17:40.093 INFO:tasks.ceph.mds.a.smithi032.stderr: -4480> 2023-10-16T08:16:26.335+0000 7f18d54bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:17:40.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T08:17:40.144 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-10-16T08:17:40.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa0943d9640 1 Processor -- start 2023-10-16T08:17:40.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa0943d9640 1 -- start start 2023-10-16T08:17:40.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa0943d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08c06a5a0 0x7fa08c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa0943d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c0b4370 con 0x7fa08c0b3f00 2023-10-16T08:17:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa0943d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c0b4600 con 0x7fa08c159900 2023-10-16T08:17:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa0943d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08c0cd740 con 0x7fa08c06a5a0 2023-10-16T08:17:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa091974640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08c06a5a0 0x7fa08c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa092976640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa08c0b3f00 0x7fa08c0c9b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42290/0 (socket says 172.21.15.32:42290) 2023-10-16T08:17:40.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa092976640 1 -- 172.21.15.32:0/277502917 learned_addr learned my addr 172.21.15.32:0/277502917 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa091974640 1 -- 172.21.15.32:0/277502917 >> v1:172.21.15.186:6789/0 conn(0x7fa08c159900 legacy=0x7fa08c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa091974640 1 -- 172.21.15.32:0/277502917 >> v1:172.21.15.32:6789/0 conn(0x7fa08c0b3f00 legacy=0x7fa08c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.249+0000 7fa091974640 1 -- 172.21.15.32:0/277502917 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c0c8dd0 con 0x7fa08c06a5a0 2023-10-16T08:17:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa091974640 1 --2- 172.21.15.32:0/277502917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c06a5a0 0x7fa08c06e260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00ba60 tx=0x7fa07c01eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a04a56d462620b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa091173640 1 -- 172.21.15.32:0/277502917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0a7bf0 con 0x7fa08c06a5a0 2023-10-16T08:17:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa091173640 1 -- 172.21.15.32:0/277502917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c0a7d50 con 0x7fa08c06a5a0 2023-10-16T08:17:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa091173640 1 -- 172.21.15.32:0/277502917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0af660 con 0x7fa08c06a5a0 2023-10-16T08:17:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 -- 172.21.15.32:0/277502917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c06a5a0 msgr2=0x7fa08c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 --2- 172.21.15.32:0/277502917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c06a5a0 0x7fa08c06e260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00ba60 tx=0x7fa07c01eb20 comp rx=0 tx=0).stop 2023-10-16T08:17:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 -- 172.21.15.32:0/277502917 shutdown_connections 2023-10-16T08:17:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 --2- 172.21.15.32:0/277502917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c06a5a0 0x7fa08c06e260 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 -- 172.21.15.32:0/277502917 >> 172.21.15.32:0/277502917 conn(0x7fa08c0b0da0 msgr2=0x7fa08c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 -- 172.21.15.32:0/277502917 shutdown_connections 2023-10-16T08:17:40.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 -- 172.21.15.32:0/277502917 wait complete. 2023-10-16T08:17:40.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 Processor -- start 2023-10-16T08:17:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.253+0000 7fa0943d9640 1 -- start start 2023-10-16T08:17:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa08c06a5a0 0x7fa08c1f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa08c0b3f00 0x7fa08c14a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 0x7fa08c14e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c1fc980 con 0x7fa08c159900 2023-10-16T08:17:40.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c1f8fa0 con 0x7fa08c0b3f00 2023-10-16T08:17:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c14ac60 con 0x7fa08c06a5a0 2023-10-16T08:17:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 0x7fa08c14e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 0x7fa08c14e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59640/0 (socket says 172.21.15.32:59640) 2023-10-16T08:17:40.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 -- 172.21.15.32:0/1718142194 learned_addr learned my addr 172.21.15.32:0/1718142194 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092175640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa08c06a5a0 0x7fa08c1f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa091974640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa08c0b3f00 0x7fa08c14a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 -- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa08c06a5a0 msgr2=0x7fa08c1f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa08c06a5a0 0x7fa08c1f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 -- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa08c0b3f00 msgr2=0x7fa08c14a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa08c0b3f00 0x7fa08c14a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 -- 172.21.15.32:0/1718142194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c00b6c0 con 0x7fa08c159900 2023-10-16T08:17:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa092976640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 0x7fa08c14e100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa08400cbc0 tx=0x7fa08400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:40.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400ad40 con 0x7fa08c159900 2023-10-16T08:17:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084012400 con 0x7fa08c159900 2023-10-16T08:17:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400aea0 con 0x7fa08c159900 2023-10-16T08:17:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c14d6d0 con 0x7fa08c159900 2023-10-16T08:17:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c15bb10 con 0x7fa08c159900 2023-10-16T08:17:40.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.257+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08c04e170 con 0x7fa08c159900 2023-10-16T08:17:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.261+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa084007070 con 0x7fa08c159900 2023-10-16T08:17:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.261+0000 7fa082ffd640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa0700664a0 0x7fa070068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.261+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fa0840840f0 con 0x7fa08c159900 2023-10-16T08:17:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.261+0000 7fa092175640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa0700664a0 0x7fa070068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:40.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.261+0000 7fa092175640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa0700664a0 0x7fa070068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa08c1fd6f0 tx=0x7fa088009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:40.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.265+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa084051200 con 0x7fa08c159900 2023-10-16T08:17:40.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:40.457+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa08c0ce820 con 0x7fa08c159900 2023-10-16T08:17:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.077+0000 7fa082ffd640 1 -- 172.21.15.32:0/1718142194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa084054820 con 0x7fa08c159900 2023-10-16T08:17:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa0700664a0 msgr2=0x7fa070068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa0700664a0 0x7fa070068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa08c1fd6f0 tx=0x7fa088009040 comp rx=0 tx=0).stop 2023-10-16T08:17:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 msgr2=0x7fa08c14e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 0x7fa08c14e100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa08400cbc0 tx=0x7fa08400a430 comp rx=0 tx=0).stop 2023-10-16T08:17:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 shutdown_connections 2023-10-16T08:17:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c159900 0x7fa08c14e100 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa08c0b3f00 0x7fa08c14a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa0700664a0 0x7fa070068960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 --2- 172.21.15.32:0/1718142194 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa08c06a5a0 0x7fa08c1f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 >> 172.21.15.32:0/1718142194 conn(0x7fa08c0b0da0 msgr2=0x7fa08c0701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 shutdown_connections 2023-10-16T08:17:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.081+0000 7fa0943d9640 1 -- 172.21.15.32:0/1718142194 wait complete. 2023-10-16T08:17:41.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T08:17:41.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.301+0000 7fd15f29c640 1 Processor -- start 2023-10-16T08:17:41.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.301+0000 7fd15f29c640 1 -- start start 2023-10-16T08:17:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15f29c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd1580cd390 0x7fd1580cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15f29c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1580cb900 0x7fd1580ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15f29c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd770 0x7fd1580ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15f29c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580cbe60 con 0x7fd1580cb900 2023-10-16T08:17:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15f29c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580cc140 con 0x7fd1580cd390 2023-10-16T08:17:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15f29c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580cc420 con 0x7fd1580cd770 2023-10-16T08:17:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1580cb900 0x7fd1580ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1580cb900 0x7fd1580ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59656/0 (socket says 172.21.15.32:59656) 2023-10-16T08:17:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 -- 172.21.15.32:0/1717638094 learned_addr learned my addr 172.21.15.32:0/1717638094 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15d038640 1 --2- 172.21.15.32:0/1717638094 >> v2:172.21.15.186:3300/0 conn(0x7fd1580cd390 0x7fd1580cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15d839640 1 --2- 172.21.15.32:0/1717638094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd770 0x7fd1580ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 -- 172.21.15.32:0/1717638094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd770 msgr2=0x7fd1580ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 --2- 172.21.15.32:0/1717638094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd770 0x7fd1580ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 -- 172.21.15.32:0/1717638094 >> v2:172.21.15.186:3300/0 conn(0x7fd1580cd390 msgr2=0x7fd1580cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 --2- 172.21.15.32:0/1717638094 >> v2:172.21.15.186:3300/0 conn(0x7fd1580cd390 0x7fd1580cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.305+0000 7fd15c837640 1 -- 172.21.15.32:0/1717638094 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580cc700 con 0x7fd1580cb900 2023-10-16T08:17:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15c837640 1 --2- 172.21.15.32:0/1717638094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd1580ce310 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd158066e30 tx=0x7fd14801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb726d17bcf2d9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd14ffff640 1 -- 172.21.15.32:0/1717638094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1480a89b0 con 0x7fd1580cb900 2023-10-16T08:17:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd14ffff640 1 -- 172.21.15.32:0/1717638094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1480a8b10 con 0x7fd1580cb900 2023-10-16T08:17:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd14ffff640 1 -- 172.21.15.32:0/1717638094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1480a8ea0 con 0x7fd1580cb900 2023-10-16T08:17:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 -- 172.21.15.32:0/1717638094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 msgr2=0x7fd1580ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 --2- 172.21.15.32:0/1717638094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd1580ce310 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd158066e30 tx=0x7fd14801ecf0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 -- 172.21.15.32:0/1717638094 shutdown_connections 2023-10-16T08:17:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 --2- 172.21.15.32:0/1717638094 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd770 0x7fd1580ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 --2- 172.21.15.32:0/1717638094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd1580ce310 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 --2- 172.21.15.32:0/1717638094 >> v2:172.21.15.186:3300/0 conn(0x7fd1580cd390 0x7fd1580cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 -- 172.21.15.32:0/1717638094 >> 172.21.15.32:0/1717638094 conn(0x7fd158089f80 msgr2=0x7fd1580d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 -- 172.21.15.32:0/1717638094 shutdown_connections 2023-10-16T08:17:41.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 -- 172.21.15.32:0/1717638094 wait complete. 2023-10-16T08:17:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 Processor -- start 2023-10-16T08:17:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.309+0000 7fd15f29c640 1 -- start start 2023-10-16T08:17:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd158155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd390 0x7fd1580c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1580cd770 0x7fd1580c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580c69e0 con 0x7fd1580cb900 2023-10-16T08:17:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580c8980 con 0x7fd1580cd770 2023-10-16T08:17:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15815a6a0 con 0x7fd1580cd390 2023-10-16T08:17:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd158155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd158155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59672/0 (socket says 172.21.15.32:59672) 2023-10-16T08:17:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 -- 172.21.15.32:0/305147804 learned_addr learned my addr 172.21.15.32:0/305147804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15c837640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd390 0x7fd1580c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d839640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1580cd770 0x7fd1580c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 -- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd390 msgr2=0x7fd1580c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd390 0x7fd1580c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 -- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1580cd770 msgr2=0x7fd1580c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1580cd770 0x7fd1580c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 -- 172.21.15.32:0/305147804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480094d0 con 0x7fd1580cb900 2023-10-16T08:17:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15d038640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd158155620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd15400d800 tx=0x7fd15400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15400b6c0 con 0x7fd1580cb900 2023-10-16T08:17:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15400b820 con 0x7fd1580cb900 2023-10-16T08:17:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15400bb90 con 0x7fd1580cb900 2023-10-16T08:17:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd158159d00 con 0x7fd1580cb900 2023-10-16T08:17:41.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15815a2c0 con 0x7fd1580cb900 2023-10-16T08:17:41.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15815a910 con 0x7fd1580cb900 2023-10-16T08:17:41.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.313+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd15400bcf0 con 0x7fd1580cb900 2023-10-16T08:17:41.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.317+0000 7fd14dffb640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd13c0664a0 0x7fd13c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:41.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.317+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fd154084820 con 0x7fd1580cb900 2023-10-16T08:17:41.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.317+0000 7fd15c837640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd13c0664a0 0x7fd13c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:41.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.317+0000 7fd15c837640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd13c0664a0 0x7fd13c068960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd1480b2000 tx=0x7fd148008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:41.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.321+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd154051890 con 0x7fd1580cb900 2023-10-16T08:17:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:41.477+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd158073fa0 con 0x7fd1580cb900 2023-10-16T08:17:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.077+0000 7fd14dffb640 1 -- 172.21.15.32:0/305147804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd154011340 con 0x7fd1580cb900 2023-10-16T08:17:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-10-16T08:17:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.081+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd13c0664a0 msgr2=0x7fd13c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.081+0000 7fd15f29c640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd13c0664a0 0x7fd13c068960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd1480b2000 tx=0x7fd148008040 comp rx=0 tx=0).stop 2023-10-16T08:17:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 msgr2=0x7fd158155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd158155620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd15400d800 tx=0x7fd15400dcc0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 shutdown_connections 2023-10-16T08:17:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1580cd770 0x7fd1580c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd13c0664a0 0x7fd13c068960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd1580cd390 0x7fd1580c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 --2- 172.21.15.32:0/305147804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1580cb900 0x7fd158155620 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 >> 172.21.15.32:0/305147804 conn(0x7fd158089f80 msgr2=0x7fd1581e8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.085+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 shutdown_connections 2023-10-16T08:17:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.089+0000 7fd15f29c640 1 -- 172.21.15.32:0/305147804 wait complete. 2023-10-16T08:17:42.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T08:17:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 Processor -- start 2023-10-16T08:17:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 -- start start 2023-10-16T08:17:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4f01600e0 0x7fd4f00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f00cfd50 0x7fd4f00cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00cdc40 con 0x7fd4f00cd7f0 2023-10-16T08:17:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f00caf20 con 0x7fd4f01600e0 2023-10-16T08:17:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f7646640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00b3720 con 0x7fd4f00cfd50 2023-10-16T08:17:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f4be1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f00cfd50 0x7fd4f00cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4f01600e0 0x7fd4f00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd4f01600e0 0x7fd4f00cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:43264/0 (socket says 172.21.15.32:43264) 2023-10-16T08:17:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 -- 172.21.15.32:0/3351982639 learned_addr learned my addr 172.21.15.32:0/3351982639 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f4be1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f00cfd50 0x7fd4f00cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:56846/0 (socket says 172.21.15.32:56846) 2023-10-16T08:17:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f5be3640 1 --1- 172.21.15.32:0/3351982639 >> v1:172.21.15.186:6789/0 conn(0x7fd4f00cd7f0 0x7fd4f006b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:39970/0 (socket says 172.21.15.32:39970) 2023-10-16T08:17:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 -- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f00cfd50 msgr2=0x7fd4f00cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 --2- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f00cfd50 0x7fd4f00cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 -- 172.21.15.32:0/3351982639 >> v1:172.21.15.186:6789/0 conn(0x7fd4f00cd7f0 legacy=0x7fd4f006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.269+0000 7fd4f53e2640 1 -- 172.21.15.32:0/3351982639 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00d02a0 con 0x7fd4f01600e0 2023-10-16T08:17:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f53e2640 1 --2- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f01600e0 0x7fd4f00cf810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009870 tx=0x7fd4ec01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d855061b0ab70332 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4e7fff640 1 -- 172.21.15.32:0/3351982639 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0ac600 con 0x7fd4f01600e0 2023-10-16T08:17:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4e7fff640 1 -- 172.21.15.32:0/3351982639 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec0a8050 con 0x7fd4f01600e0 2023-10-16T08:17:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4e7fff640 1 -- 172.21.15.32:0/3351982639 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0ac980 con 0x7fd4f01600e0 2023-10-16T08:17:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 -- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f01600e0 msgr2=0x7fd4f00cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 --2- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f01600e0 0x7fd4f00cf810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009870 tx=0x7fd4ec01c480 comp rx=0 tx=0).stop 2023-10-16T08:17:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 -- 172.21.15.32:0/3351982639 shutdown_connections 2023-10-16T08:17:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 --2- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f00cfd50 0x7fd4f00cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 --2- 172.21.15.32:0/3351982639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f01600e0 0x7fd4f00cf810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 -- 172.21.15.32:0/3351982639 >> 172.21.15.32:0/3351982639 conn(0x7fd4f00d08d0 msgr2=0x7fd4f00d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 -- 172.21.15.32:0/3351982639 shutdown_connections 2023-10-16T08:17:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.273+0000 7fd4f7646640 1 -- 172.21.15.32:0/3351982639 wait complete. 2023-10-16T08:17:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 Processor -- start 2023-10-16T08:17:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 -- start start 2023-10-16T08:17:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f00cd7f0 0x7fd4f00c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 0x7fd4f014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f01600e0 0x7fd4f00c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f014dbf0 con 0x7fd4f00cfd50 2023-10-16T08:17:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00b4b80 con 0x7fd4f00cd7f0 2023-10-16T08:17:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00c4ba0 con 0x7fd4f01600e0 2023-10-16T08:17:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 0x7fd4f014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 0x7fd4f014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59674/0 (socket says 172.21.15.32:59674) 2023-10-16T08:17:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 -- 172.21.15.32:0/1957069420 learned_addr learned my addr 172.21.15.32:0/1957069420 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f53e2640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f00cd7f0 0x7fd4f00c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f5be3640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f01600e0 0x7fd4f00c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 -- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f01600e0 msgr2=0x7fd4f00c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f01600e0 0x7fd4f00c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 -- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f00cd7f0 msgr2=0x7fd4f00c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f00cd7f0 0x7fd4f00c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 -- 172.21.15.32:0/1957069420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0094d0 con 0x7fd4f00cfd50 2023-10-16T08:17:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f5be3640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f01600e0 0x7fd4f00c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f4be1640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 0x7fd4f014d6b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e930 tx=0x7fd4e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0009690 con 0x7fd4f00cfd50 2023-10-16T08:17:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000ca00 con 0x7fd4f00cfd50 2023-10-16T08:17:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cd70 con 0x7fd4f00cfd50 2023-10-16T08:17:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f00c4e80 con 0x7fd4f00cfd50 2023-10-16T08:17:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.277+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f00c5440 con 0x7fd4f00cfd50 2023-10-16T08:17:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.281+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f00b4d80 con 0x7fd4f00cfd50 2023-10-16T08:17:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.281+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd4e0013470 con 0x7fd4f00cfd50 2023-10-16T08:17:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.281+0000 7fd4e5ffb640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4d40664a0 0x7fd4d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.281+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fd4e00843a0 con 0x7fd4f00cfd50 2023-10-16T08:17:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.281+0000 7fd4f53e2640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4d40664a0 0x7fd4d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.281+0000 7fd4f53e2640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4d40664a0 0x7fd4d4068960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0ae000 tx=0x7fd4ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.285+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd4f00b4d80 con 0x7fd4f00cfd50 2023-10-16T08:17:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:42.449+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4f00b4d50 con 0x7fd4f00cfd50 2023-10-16T08:17:43.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.081+0000 7fd4e5ffb640 1 -- 172.21.15.32:0/1957069420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd4e0051690 con 0x7fd4f00cfd50 2023-10-16T08:17:43.088 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-10-16T08:17:43.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.085+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4d40664a0 msgr2=0x7fd4d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.085+0000 7fd4f7646640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4d40664a0 0x7fd4d4068960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0ae000 tx=0x7fd4ec0a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:43.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.085+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 msgr2=0x7fd4f014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.085+0000 7fd4f7646640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 0x7fd4f014d6b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e930 tx=0x7fd4e000edf0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 shutdown_connections 2023-10-16T08:17:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4d40664a0 0x7fd4d4068960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd4f01600e0 0x7fd4f00c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4f00cfd50 0x7fd4f014d6b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 --2- 172.21.15.32:0/1957069420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f00cd7f0 0x7fd4f00c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 >> 172.21.15.32:0/1957069420 conn(0x7fd4f00d08d0 msgr2=0x7fd4f006fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 shutdown_connections 2023-10-16T08:17:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.089+0000 7fd4f7646640 1 -- 172.21.15.32:0/1957069420 wait complete. 2023-10-16T08:17:43.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-16T08:17:43.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e88594640 1 Processor -- start 2023-10-16T08:17:43.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e88594640 1 -- start start 2023-10-16T08:17:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e88594640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e801595a0 0x7f8e8006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e88594640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800c8a20 con 0x7f8e80159980 2023-10-16T08:17:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e88594640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800c8c10 con 0x7f8e800b42d0 2023-10-16T08:17:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e88594640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800c8ef0 con 0x7f8e801595a0 2023-10-16T08:17:43.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e85b2f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8e80159980 0x7f8e800b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59416/0 (socket says 172.21.15.32:59416) 2023-10-16T08:17:43.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e85b2f640 1 -- 172.21.15.32:0/2632271916 learned_addr learned my addr 172.21.15.32:0/2632271916 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e86330640 1 --2- 172.21.15.32:0/2632271916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e801595a0 0x7f8e8006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922017914 0 0) 0x7f8e800c8a20 con 0x7f8e80159980 2023-10-16T08:17:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e6c003530 con 0x7f8e80159980 2023-10-16T08:17:43.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267632257 0 0) 0x7f8e800c8c10 con 0x7f8e800b42d0 2023-10-16T08:17:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e800c8a20 con 0x7f8e800b42d0 2023-10-16T08:17:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927318751 0 0) 0x7f8e6c003530 con 0x7f8e80159980 2023-10-16T08:17:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e86330640 1 -- 172.21.15.32:0/2632271916 >> v1:172.21.15.186:6789/0 conn(0x7f8e800b42d0 legacy=0x7f8e8015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e86330640 1 -- 172.21.15.32:0/2632271916 >> v1:172.21.15.32:6789/0 conn(0x7f8e80159980 legacy=0x7f8e800b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.269+0000 7f8e86330640 1 -- 172.21.15.32:0/2632271916 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e800c91d0 con 0x7f8e801595a0 2023-10-16T08:17:43.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e86330640 1 --2- 172.21.15.32:0/2632271916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e801595a0 0x7f8e8006c4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097f0 tx=0x7f8e7c01c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca422bb653bc192 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:43.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70001600 con 0x7f8e801595a0 2023-10-16T08:17:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0a7050 con 0x7f8e801595a0 2023-10-16T08:17:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e8532e640 1 -- 172.21.15.32:0/2632271916 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0ab600 con 0x7f8e801595a0 2023-10-16T08:17:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 -- 172.21.15.32:0/2632271916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e801595a0 msgr2=0x7f8e8006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 --2- 172.21.15.32:0/2632271916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e801595a0 0x7f8e8006c4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097f0 tx=0x7f8e7c01c320 comp rx=0 tx=0).stop 2023-10-16T08:17:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 -- 172.21.15.32:0/2632271916 shutdown_connections 2023-10-16T08:17:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 --2- 172.21.15.32:0/2632271916 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e801595a0 0x7f8e8006c4b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 -- 172.21.15.32:0/2632271916 >> 172.21.15.32:0/2632271916 conn(0x7f8e8006a820 msgr2=0x7f8e800b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 -- 172.21.15.32:0/2632271916 shutdown_connections 2023-10-16T08:17:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.273+0000 7f8e88594640 1 -- 172.21.15.32:0/2632271916 wait complete. 2023-10-16T08:17:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 Processor -- start 2023-10-16T08:17:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 -- start start 2023-10-16T08:17:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e800b42d0 0x7f8e80147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 0x7f8e8015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e8015af90 0x7f8e8015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801f9680 con 0x7f8e801595a0 2023-10-16T08:17:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80152950 con 0x7f8e8015af90 2023-10-16T08:17:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801476a0 con 0x7f8e800b42d0 2023-10-16T08:17:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 0x7f8e8015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 0x7f8e8015b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50440/0 (socket says 172.21.15.32:50440) 2023-10-16T08:17:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 -- 172.21.15.32:0/224266631 learned_addr learned my addr 172.21.15.32:0/224266631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e86330640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e800b42d0 0x7f8e80147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e86b31640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e8015af90 0x7f8e8015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 -- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e800b42d0 msgr2=0x7f8e80147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e800b42d0 0x7f8e80147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 -- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e8015af90 msgr2=0x7f8e8015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e8015af90 0x7f8e8015bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 -- 172.21.15.32:0/224266631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009420 con 0x7f8e801595a0 2023-10-16T08:17:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e85b2f640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 0x7f8e8015b700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c700 tx=0x7f8e7000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70005ae0 con 0x7f8e801595a0 2023-10-16T08:17:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e80152020 con 0x7f8e801595a0 2023-10-16T08:17:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e70005c40 con 0x7f8e801595a0 2023-10-16T08:17:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70007070 con 0x7f8e801595a0 2023-10-16T08:17:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.277+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801525e0 con 0x7f8e801595a0 2023-10-16T08:17:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.281+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e80152ac0 con 0x7f8e801595a0 2023-10-16T08:17:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.281+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8e700071d0 con 0x7f8e801595a0 2023-10-16T08:17:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.281+0000 7f8e76ffd640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e640664a0 0x7f8e64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.281+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f8e70083b80 con 0x7f8e801595a0 2023-10-16T08:17:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.281+0000 7f8e86330640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e640664a0 0x7f8e64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.281+0000 7f8e86330640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e640664a0 0x7f8e64068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0ad000 tx=0x7f8e7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:43.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.285+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8e70050f20 con 0x7f8e801595a0 2023-10-16T08:17:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:43.441+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8e8004e170 con 0x7f8e801595a0 2023-10-16T08:17:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.097+0000 7f8e76ffd640 1 -- 172.21.15.32:0/224266631 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8e70083e60 con 0x7f8e801595a0 2023-10-16T08:17:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data_ec' removed 2023-10-16T08:17:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.101+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e640664a0 msgr2=0x7f8e64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.101+0000 7f8e88594640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e640664a0 0x7f8e64068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0ad000 tx=0x7f8e7c008040 comp rx=0 tx=0).stop 2023-10-16T08:17:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.101+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 msgr2=0x7f8e8015b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.101+0000 7f8e88594640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 0x7f8e8015b700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c700 tx=0x7f8e7000cbc0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 shutdown_connections 2023-10-16T08:17:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e8015af90 0x7f8e8015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e801595a0 0x7f8e8015b700 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c700 tx=0x7f8e7000cbc0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8e640664a0 0x7f8e64068960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 --2- 172.21.15.32:0/224266631 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8e800b42d0 0x7f8e80147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 >> 172.21.15.32:0/224266631 conn(0x7f8e8006a820 msgr2=0x7f8e800cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 shutdown_connections 2023-10-16T08:17:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.105+0000 7f8e88594640 1 -- 172.21.15.32:0/224266631 wait complete. 2023-10-16T08:17:44.145 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T08:17:44.145 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T08:17:44.146 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T08:17:44.146 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:44.146 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T08:17:44.146 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T08:17:44.147 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T08:17:44.147 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:44.148 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T08:17:44.148 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T08:17:44.148 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T08:17:44.149 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:44.149 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T08:17:44.149 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T08:17:44.150 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T08:17:44.150 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:17:44.160 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T08:17:44.161 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T08:17:44.162 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T08:17:44.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T08:17:44.164 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T08:17:44.165 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T08:17:44.166 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T08:17:44.166 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T08:17:44.208 INFO:tasks.ceph.mds.a:Started 2023-10-16T08:17:44.209 DEBUG:teuthology.parallel:result is None 2023-10-16T08:17:44.211 INFO:tasks.ceph.mds.c:Started 2023-10-16T08:17:44.211 DEBUG:teuthology.parallel:result is None 2023-10-16T08:17:44.215 INFO:tasks.ceph.mds.b:Started 2023-10-16T08:17:44.215 DEBUG:teuthology.parallel:result is None 2023-10-16T08:17:44.217 INFO:tasks.ceph.mds.d:Started 2023-10-16T08:17:44.217 DEBUG:teuthology.parallel:result is None 2023-10-16T08:17:44.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T08:17:44.296 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-10-16T08:17:44.300 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-10-16T08:17:44.303 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-10-16T08:17:44.303 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-10-16T08:17:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 Processor -- start 2023-10-16T08:17:44.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 -- start start 2023-10-16T08:17:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13b415cb90 0x7f13b40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f13b40d15d0 0x7f13b40c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b40ce770 con 0x7f13b40d3010 2023-10-16T08:17:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40d0600 con 0x7f13b415cb90 2023-10-16T08:17:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b8d72640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40c6450 con 0x7f13b40d15d0 2023-10-16T08:17:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13b415cb90 0x7f13b40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13b415cb90 0x7f13b40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50498/0 (socket says 172.21.15.32:50498) 2023-10-16T08:17:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 -- 172.21.15.32:0/276373683 learned_addr learned my addr 172.21.15.32:0/276373683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13abfff640 1 --1- 172.21.15.32:0/276373683 >> v1:172.21.15.32:6789/0 conn(0x7f13b40d3010 0x7f13b40d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59438/0 (socket says 172.21.15.32:59438) 2023-10-16T08:17:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/276373683 >> v2:172.21.15.186:3300/0 conn(0x7f13b40d15d0 0x7f13b40c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 -- 172.21.15.32:0/276373683 >> v2:172.21.15.186:3300/0 conn(0x7f13b40d15d0 msgr2=0x7f13b40c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 --2- 172.21.15.32:0/276373683 >> v2:172.21.15.186:3300/0 conn(0x7f13b40d15d0 0x7f13b40c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 -- 172.21.15.32:0/276373683 >> v1:172.21.15.32:6789/0 conn(0x7f13b40d3010 legacy=0x7f13b40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b259c640 1 -- 172.21.15.32:0/276373683 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b40d08a0 con 0x7f13b415cb90 2023-10-16T08:17:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.361+0000 7f13b1d9b640 1 -- 172.21.15.32:0/276373683 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465451673 0 0) 0x7f13b40ce770 con 0x7f13b40d3010 2023-10-16T08:17:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b259c640 1 --2- 172.21.15.32:0/276373683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40cfb10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13b40ccee0 tx=0x7f13a401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8186dd237518f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b1d9b640 1 -- 172.21.15.32:0/276373683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40ab640 con 0x7f13b415cb90 2023-10-16T08:17:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b1d9b640 1 -- 172.21.15.32:0/276373683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a40a7070 con 0x7f13b415cb90 2023-10-16T08:17:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b1d9b640 1 -- 172.21.15.32:0/276373683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40ab9c0 con 0x7f13b415cb90 2023-10-16T08:17:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 -- 172.21.15.32:0/276373683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 msgr2=0x7f13b40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 --2- 172.21.15.32:0/276373683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40cfb10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13b40ccee0 tx=0x7f13a401c440 comp rx=0 tx=0).stop 2023-10-16T08:17:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 -- 172.21.15.32:0/276373683 shutdown_connections 2023-10-16T08:17:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 --2- 172.21.15.32:0/276373683 >> v2:172.21.15.186:3300/0 conn(0x7f13b40d15d0 0x7f13b40c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 --2- 172.21.15.32:0/276373683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40cfb10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 -- 172.21.15.32:0/276373683 >> 172.21.15.32:0/276373683 conn(0x7f13b409bc50 msgr2=0x7f13b40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 -- 172.21.15.32:0/276373683 shutdown_connections 2023-10-16T08:17:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.365+0000 7f13b8d72640 1 -- 172.21.15.32:0/276373683 wait complete. 2023-10-16T08:17:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 Processor -- start 2023-10-16T08:17:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 -- start start 2023-10-16T08:17:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13b40d15d0 0x7f13b41539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13b40d3010 0x7f13b40b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b415af40 con 0x7f13b415cb90 2023-10-16T08:17:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b415a330 con 0x7f13b40d3010 2023-10-16T08:17:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b415a610 con 0x7f13b40d15d0 2023-10-16T08:17:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50510/0 (socket says 172.21.15.32:50510) 2023-10-16T08:17:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b259c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13b40d15d0 0x7f13b41539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 -- 172.21.15.32:0/4271334415 learned_addr learned my addr 172.21.15.32:0/4271334415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:44.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13abfff640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13b40d3010 0x7f13b40b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 -- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13b40d15d0 msgr2=0x7f13b41539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13b40d15d0 0x7f13b41539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 -- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13b40d3010 msgr2=0x7f13b40b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13b40d3010 0x7f13b40b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 -- 172.21.15.32:0/4271334415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009480 con 0x7f13b415cb90 2023-10-16T08:17:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b2d9d640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40b7bc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f13ac007e40 tx=0x7f13ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac014620 con 0x7f13b415cb90 2023-10-16T08:17:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac00ee90 con 0x7f13b415cb90 2023-10-16T08:17:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0149e0 con 0x7f13b415cb90 2023-10-16T08:17:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b415a8f0 con 0x7f13b415cb90 2023-10-16T08:17:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.369+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4159fc0 con 0x7f13b415cb90 2023-10-16T08:17:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.373+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b40ce1d0 con 0x7f13b415cb90 2023-10-16T08:17:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.373+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13ac014b40 con 0x7f13b415cb90 2023-10-16T08:17:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.373+0000 7f13aaffd640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f13940664a0 0x7f1394068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.373+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f13ac084530 con 0x7f13b415cb90 2023-10-16T08:17:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.373+0000 7f13b259c640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f13940664a0 0x7f1394068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.373+0000 7f13b259c640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f13940664a0 0x7f1394068960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13a40af000 tx=0x7f13a40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:44.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.377+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f13ac051c20 con 0x7f13b415cb90 2023-10-16T08:17:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:44.549+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f13b4074010 con 0x7f13b415cb90 2023-10-16T08:17:45.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.109+0000 7f13aaffd640 1 -- 172.21.15.32:0/4271334415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f13ac055240 con 0x7f13b415cb90 2023-10-16T08:17:45.115 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-10-16T08:17:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.109+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f13940664a0 msgr2=0x7f1394068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.109+0000 7f13b8d72640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f13940664a0 0x7f1394068960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13a40af000 tx=0x7f13a40a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 msgr2=0x7f13b40b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40b7bc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f13ac007e40 tx=0x7f13ac00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 shutdown_connections 2023-10-16T08:17:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13b415cb90 0x7f13b40b7bc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13b40d3010 0x7f13b40b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f13940664a0 0x7f1394068960 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13a40af000 tx=0x7f13a40a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 --2- 172.21.15.32:0/4271334415 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13b40d15d0 0x7f13b41539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 >> 172.21.15.32:0/4271334415 conn(0x7f13b409bc50 msgr2=0x7f13b406c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 shutdown_connections 2023-10-16T08:17:45.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.113+0000 7f13b8d72640 1 -- 172.21.15.32:0/4271334415 wait complete. 2023-10-16T08:17:45.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T08:17:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 Processor -- start 2023-10-16T08:17:45.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 -- start start 2023-10-16T08:17:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fcc0d3170 0x7f0fcc0d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc0cefe0 0x7f0fcc0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc0ce730 con 0x7f0fcc15c910 2023-10-16T08:17:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0cf9d0 con 0x7f0fcc0d3170 2023-10-16T08:17:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2867640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0ce540 con 0x7f0fcc0cefe0 2023-10-16T08:17:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1865640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fcc15c910 0x7f0fcc0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59442/0 (socket says 172.21.15.32:59442) 2023-10-16T08:17:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1865640 1 -- 172.21.15.32:0/1216780822 learned_addr learned my addr 172.21.15.32:0/1216780822 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1064640 1 --2- 172.21.15.32:0/1216780822 >> v2:172.21.15.32:3300/0 conn(0x7f0fcc0d3170 0x7f0fcc0d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1064640 1 -- 172.21.15.32:0/1216780822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc0cefe0 msgr2=0x7f0fcc0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2066640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc0cefe0 0x7f0fcc0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1064640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc0cefe0 0x7f0fcc0cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1064640 1 -- 172.21.15.32:0/1216780822 >> v1:172.21.15.32:6789/0 conn(0x7f0fcc15c910 legacy=0x7f0fcc0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd1064640 1 -- 172.21.15.32:0/1216780822 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0d0dd0 con 0x7f0fcc0d3170 2023-10-16T08:17:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd0863640 1 -- 172.21.15.32:0/1216780822 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144866740 0 0) 0x7f0fcc0ce730 con 0x7f0fcc15c910 2023-10-16T08:17:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.285+0000 7f0fd2066640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc0cefe0 0x7f0fcc0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd1064640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc0d1260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097f0 tx=0x7f0fbc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b022895e97c30c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd0863640 1 -- 172.21.15.32:0/1216780822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0ab600 con 0x7f0fcc0d3170 2023-10-16T08:17:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd0863640 1 -- 172.21.15.32:0/1216780822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc0a7050 con 0x7f0fcc0d3170 2023-10-16T08:17:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd0863640 1 -- 172.21.15.32:0/1216780822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0ab980 con 0x7f0fcc0d3170 2023-10-16T08:17:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 -- 172.21.15.32:0/1216780822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 msgr2=0x7f0fcc0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc0d1260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097f0 tx=0x7f0fbc01c3f0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 -- 172.21.15.32:0/1216780822 shutdown_connections 2023-10-16T08:17:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc0cefe0 0x7f0fcc0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 --2- 172.21.15.32:0/1216780822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc0d1260 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 -- 172.21.15.32:0/1216780822 >> 172.21.15.32:0/1216780822 conn(0x7f0fcc09bb50 msgr2=0x7f0fcc0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 -- 172.21.15.32:0/1216780822 shutdown_connections 2023-10-16T08:17:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 -- 172.21.15.32:0/1216780822 wait complete. 2023-10-16T08:17:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.289+0000 7f0fd2867640 1 Processor -- start 2023-10-16T08:17:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- start start 2023-10-16T08:17:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc0cefe0 0x7f0fcc14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc15c910 0x7f0fcc153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc159760 con 0x7f0fcc0d3170 2023-10-16T08:17:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc14f1a0 con 0x7f0fcc0cefe0 2023-10-16T08:17:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1599b0 con 0x7f0fcc15c910 2023-10-16T08:17:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50532/0 (socket says 172.21.15.32:50532) 2023-10-16T08:17:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 -- 172.21.15.32:0/628014530 learned_addr learned my addr 172.21.15.32:0/628014530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2066640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc15c910 0x7f0fcc153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1865640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc0cefe0 0x7f0fcc14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 -- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc15c910 msgr2=0x7f0fcc153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc15c910 0x7f0fcc153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 -- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc0cefe0 msgr2=0x7f0fcc14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc0cefe0 0x7f0fcc14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 -- 172.21.15.32:0/628014530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc009420 con 0x7f0fcc0d3170 2023-10-16T08:17:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1865640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc0cefe0 0x7f0fcc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:45.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd1064640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc15bb50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc01a020 tx=0x7f0fbc01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0b7650 con 0x7f0fcc0d3170 2023-10-16T08:17:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc0b77d0 con 0x7f0fcc0d3170 2023-10-16T08:17:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0b7b80 con 0x7f0fcc0d3170 2023-10-16T08:17:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc159c90 con 0x7f0fcc0d3170 2023-10-16T08:17:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc15a250 con 0x7f0fcc0d3170 2023-10-16T08:17:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.293+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc04e9c0 con 0x7f0fcc0d3170 2023-10-16T08:17:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.297+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0fbc0b7ce0 con 0x7f0fcc0d3170 2023-10-16T08:17:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.297+0000 7f0fc27fc640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0fb00664a0 0x7f0fb0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.297+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0fbc11e540 con 0x7f0fcc0d3170 2023-10-16T08:17:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.297+0000 7f0fd1865640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0fb00664a0 0x7f0fb0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.297+0000 7f0fd1865640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0fb00664a0 0x7f0fb0068960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8001740 tx=0x7f0fc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.301+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0fbc0ebce0 con 0x7f0fcc0d3170 2023-10-16T08:17:45.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0fcc0cfe20 con 0x7f0fcc0d3170 2023-10-16T08:17:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fc27fc640 1 -- 172.21.15.32:0/628014530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f0fbc0ef300 con 0x7f0fcc0d3170 2023-10-16T08:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-10-16T08:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.516 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-10-16T08:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8Sxlq7jUBRAAZ/PoSPxeTxzGVyZoXvddGA==", 2023-10-16T08:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.517 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:17:45.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:17:45.518 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.519 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-10-16T08:17:45.519 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlD5VrEhAAoIJTZvkt2JVMQDjd9QNNnA==", 2023-10-16T08:17:45.519 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.519 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.519 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:17:45.519 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:17:45.520 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.520 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-10-16T08:17:45.521 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlZ8QwDRAAa6bkfiFmL7N90yyiFUBOsg==", 2023-10-16T08:17:45.521 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.521 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.521 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:17:45.521 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:17:45.521 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-10-16T08:17:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxliSn5GRAA8atnMBpEoXUyjpqQtoGWgw==", 2023-10-16T08:17:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:17:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:17:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-10-16T08:17:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAZ8SxltyuHBRAA523N7XKVdNMwSLQo24Vvvg==", 2023-10-16T08:17:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-10-16T08:17:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAc8Sxl88vjCxAAh5b4FThP+7Y7k0nYLylx5Q==", 2023-10-16T08:17:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-10-16T08:17:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAf8SxlfcEoEhAABJMIQKe43NGUqMCXCq6HEA==", 2023-10-16T08:17:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-10-16T08:17:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAi8SxlomRcGRAAIu96ZzCy18pSgsR3hD/Y7Q==", 2023-10-16T08:17:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-10-16T08:17:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAo8SxlXxEEHxAAqohCELe6In6sSjbBTHY+OQ==", 2023-10-16T08:17:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-10-16T08:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAr8SxloVNhJhAADF2erI7/queuEM5kSqqJkQ==", 2023-10-16T08:17:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-10-16T08:17:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAu8SxlRtStLhAAIzAuqOD9H7iiabi1P67nNA==", 2023-10-16T08:17:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-10-16T08:17:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAx8SxlthuoNRAAubSeLAllgyLqaRY4z4G35w==", 2023-10-16T08:17:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:17:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:17:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-10-16T08:17:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ==", 2023-10-16T08:17:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:17:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:17:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:17:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-10-16T08:17:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlA8EAKBAA7lsMEVeNC7t3xvgCLWoYcQ==", 2023-10-16T08:17:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:17:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:17:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:17:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-10-16T08:17:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlBylLJBAAXNMyfzVb5MDsSdSxVN59lg==", 2023-10-16T08:17:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:17:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:17:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:17:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-10-16T08:17:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8Sxl1dxZLhAAqhBqKEmTcorRaKo5YmmzvA==", 2023-10-16T08:17:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:17:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:17:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:17:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:17:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-10-16T08:17:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAW8SxljlvpIxAA4Re4iMDl0YMQvX5A8wqXaQ==", 2023-10-16T08:17:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-10-16T08:17:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:17:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:17:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2023-10-16T08:17:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA88SxlPo3nNxAAOiDXTmnD41NjlGv33B8j8A==", 2023-10-16T08:17:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2023-10-16T08:17:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2023-10-16T08:17:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA88Sxl5JrnNxAA6qKTQcW1eOCeVULWVkHWVA==", 2023-10-16T08:17:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-16T08:17:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2023-10-16T08:17:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA88SxlPKTnNxAA3Zz+kxbdK3wG7jVGdG2Jlw==", 2023-10-16T08:17:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2023-10-16T08:17:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2023-10-16T08:17:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA88SxlVa3nNxAAT9OcF1P71hmn+QSTWZuLzg==", 2023-10-16T08:17:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-16T08:17:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-16T08:17:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA88SxlZ7bnNxAArn+zYVWiRwA/mDrsAO/YnQ==", 2023-10-16T08:17:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-16T08:17:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2023-10-16T08:17:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA88SxlesDnNxAA0cfcDbOkO0Fkc4EvIPbr/g==", 2023-10-16T08:17:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-16T08:17:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-10-16T08:17:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlT7Q7ARAAVxrt9krDiqnEkQ0+I4duzQ==", 2023-10-16T08:17:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-10-16T08:17:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-10-16T08:17:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-10-16T08:17:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAW8SxlPtfFOBAAOXlHIg/7e75c4MipG5PHGw==", 2023-10-16T08:17:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:17:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-10-16T08:17:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-10-16T08:17:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:17:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-10-16T08:17:45.563 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:17:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0fb00664a0 msgr2=0x7f0fb0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0fb00664a0 0x7f0fb0068960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8001740 tx=0x7f0fc8009040 comp rx=0 tx=0).stop 2023-10-16T08:17:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 msgr2=0x7f0fcc15bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc15bb50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc01a020 tx=0x7f0fbc01ac30 comp rx=0 tx=0).stop 2023-10-16T08:17:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 shutdown_connections 2023-10-16T08:17:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0fb00664a0 0x7f0fb0068960 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0fcc15c910 0x7f0fcc153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc0d3170 0x7f0fcc15bb50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 --2- 172.21.15.32:0/628014530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc0cefe0 0x7f0fcc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 >> 172.21.15.32:0/628014530 conn(0x7f0fcc09bb50 msgr2=0x7f0fcc0af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 shutdown_connections 2023-10-16T08:17:45.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.509+0000 7f0fd2867640 1 -- 172.21.15.32:0/628014530 wait complete. 2023-10-16T08:17:45.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-16T08:17:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.705+0000 7f75e3a46640 1 Processor -- start 2023-10-16T08:17:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.705+0000 7f75e3a46640 1 -- start start 2023-10-16T08:17:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e3a46640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f75dc0d15d0 0x7f75dc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e3a46640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0ce770 con 0x7f75dc15cb90 2023-10-16T08:17:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e3a46640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0d0600 con 0x7f75dc0d3010 2023-10-16T08:17:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e3a46640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc0c6450 con 0x7f75dc0d15d0 2023-10-16T08:17:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e17e2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75dc15cb90 0x7f75dc0d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59448/0 (socket says 172.21.15.32:59448) 2023-10-16T08:17:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e17e2640 1 -- 172.21.15.32:0/496485376 learned_addr learned my addr 172.21.15.32:0/496485376 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e0fe1640 1 --2- 172.21.15.32:0/496485376 >> v2:172.21.15.186:3300/0 conn(0x7f75dc0d15d0 0x7f75dc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75d3fff640 1 -- 172.21.15.32:0/496485376 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377005916 0 0) 0x7f75dc0ce770 con 0x7f75dc15cb90 2023-10-16T08:17:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75d3fff640 1 -- 172.21.15.32:0/496485376 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c8003530 con 0x7f75dc15cb90 2023-10-16T08:17:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e0fe1640 1 -- 172.21.15.32:0/496485376 >> v1:172.21.15.186:6789/0 conn(0x7f75dc0d3010 legacy=0x7f75dc0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e0fe1640 1 -- 172.21.15.32:0/496485376 >> v1:172.21.15.32:6789/0 conn(0x7f75dc15cb90 legacy=0x7f75dc0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.709+0000 7f75e0fe1640 1 -- 172.21.15.32:0/496485376 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0d08a0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e0fe1640 1 --2- 172.21.15.32:0/496485376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0cfb10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0097c0 tx=0x7f75cc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2712b89fe293af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75d3fff640 1 -- 172.21.15.32:0/496485376 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0ac620 con 0x7f75dc0d15d0 2023-10-16T08:17:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75d3fff640 1 -- 172.21.15.32:0/496485376 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc0ac780 con 0x7f75dc0d15d0 2023-10-16T08:17:45.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75d3fff640 1 -- 172.21.15.32:0/496485376 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0acb10 con 0x7f75dc0d15d0 2023-10-16T08:17:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 -- 172.21.15.32:0/496485376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc0d15d0 msgr2=0x7f75dc0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 --2- 172.21.15.32:0/496485376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0cfb10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0097c0 tx=0x7f75cc01c3c0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 -- 172.21.15.32:0/496485376 shutdown_connections 2023-10-16T08:17:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 --2- 172.21.15.32:0/496485376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0cfb10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 -- 172.21.15.32:0/496485376 >> 172.21.15.32:0/496485376 conn(0x7f75dc09bc50 msgr2=0x7f75dc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 -- 172.21.15.32:0/496485376 shutdown_connections 2023-10-16T08:17:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 -- 172.21.15.32:0/496485376 wait complete. 2023-10-16T08:17:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 Processor -- start 2023-10-16T08:17:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.713+0000 7f75e3a46640 1 -- start start 2023-10-16T08:17:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75dc0d3010 0x7f75dc14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc15cb90 0x7f75dc150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0b6940 con 0x7f75dc0d15d0 2023-10-16T08:17:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0b6be0 con 0x7f75dc15cb90 2023-10-16T08:17:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0b6ec0 con 0x7f75dc0d3010 2023-10-16T08:17:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0b79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50546/0 (socket says 172.21.15.32:50546) 2023-10-16T08:17:45.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 -- 172.21.15.32:0/3692760505 learned_addr learned my addr 172.21.15.32:0/3692760505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e0fe1640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75dc0d3010 0x7f75dc14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e1fe3640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc15cb90 0x7f75dc150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 -- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75dc0d3010 msgr2=0x7f75dc14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75dc0d3010 0x7f75dc14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 -- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc15cb90 msgr2=0x7f75dc150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc15cb90 0x7f75dc150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 -- 172.21.15.32:0/3692760505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc009420 con 0x7f75dc0d15d0 2023-10-16T08:17:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e17e2640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0b79a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f75d8004b10 tx=0x7f75d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8005cf0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8005e50 con 0x7f75dc0d15d0 2023-10-16T08:17:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80072f0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0b71a0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc0b7ee0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.717+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc074010 con 0x7f75dc0d15d0 2023-10-16T08:17:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.721+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f75d8007490 con 0x7f75dc0d15d0 2023-10-16T08:17:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.721+0000 7f75d1ffb640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75c00664a0 0x7f75c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.721+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f75d8082ef0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.721+0000 7f75e0fe1640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75c00664a0 0x7f75c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:45.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.721+0000 7f75e0fe1640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75c00664a0 0x7f75c0068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0ae000 tx=0x7f75cc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:45.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.725+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f75d80540e0 con 0x7f75dc0d15d0 2023-10-16T08:17:45.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.937+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f75dc073000 con 0x7f75dc0d15d0 2023-10-16T08:17:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.937+0000 7f75d1ffb640 1 -- 172.21.15.32:0/3692760505 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f75d8017070 con 0x7f75dc0d15d0 2023-10-16T08:17:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75c00664a0 msgr2=0x7f75c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75c00664a0 0x7f75c0068960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0ae000 tx=0x7f75cc0a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 msgr2=0x7f75dc0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0b79a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f75d8004b10 tx=0x7f75d800c8e0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 shutdown_connections 2023-10-16T08:17:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75dc15cb90 0x7f75dc150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75c00664a0 0x7f75c0068960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75dc0d3010 0x7f75dc14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 --2- 172.21.15.32:0/3692760505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75dc0d15d0 0x7f75dc0b79a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 >> 172.21.15.32:0/3692760505 conn(0x7f75dc09bc50 msgr2=0x7f75dc0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 shutdown_connections 2023-10-16T08:17:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:45.941+0000 7f75e3a46640 1 -- 172.21.15.32:0/3692760505 wait complete. 2023-10-16T08:17:45.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-16T08:17:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 Processor -- start 2023-10-16T08:17:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 -- start start 2023-10-16T08:17:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d415cb90 0x7f91d40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91d40d15d0 0x7f91d40c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d40ce770 con 0x7f91d40d3010 2023-10-16T08:17:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d40d0600 con 0x7f91d40d15d0 2023-10-16T08:17:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dc7f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40c6450 con 0x7f91d415cb90 2023-10-16T08:17:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91dad95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91d40d15d0 0x7f91d40c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91d9d93640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91d40d3010 0x7f91d40d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59452/0 (socket says 172.21.15.32:59452) 2023-10-16T08:17:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.105+0000 7f91d9d93640 1 -- 172.21.15.32:0/3701151193 learned_addr learned my addr 172.21.15.32:0/3701151193 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91da594640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d415cb90 0x7f91d40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91dad95640 1 -- 172.21.15.32:0/3701151193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d415cb90 msgr2=0x7f91d40cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91dad95640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d415cb90 0x7f91d40cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91dad95640 1 -- 172.21.15.32:0/3701151193 >> v1:172.21.15.32:6789/0 conn(0x7f91d40d3010 legacy=0x7f91d40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91dad95640 1 -- 172.21.15.32:0/3701151193 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40d08a0 con 0x7f91d40d15d0 2023-10-16T08:17:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91da594640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d415cb90 0x7f91d40cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91dad95640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d40c6c10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009d90 tx=0x7f91cc01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f6f2f445b8bf03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91d9592640 1 -- 172.21.15.32:0/3701151193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0a5950 con 0x7f91d40d15d0 2023-10-16T08:17:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91d9592640 1 -- 172.21.15.32:0/3701151193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0a5ab0 con 0x7f91d40d15d0 2023-10-16T08:17:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.109+0000 7f91d9592640 1 -- 172.21.15.32:0/3701151193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0a5e40 con 0x7f91d40d15d0 2023-10-16T08:17:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- 172.21.15.32:0/3701151193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 msgr2=0x7f91d40c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d40c6c10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009d90 tx=0x7f91cc01c890 comp rx=0 tx=0).stop 2023-10-16T08:17:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- 172.21.15.32:0/3701151193 shutdown_connections 2023-10-16T08:17:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d40c6c10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/3701151193 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d415cb90 0x7f91d40cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- 172.21.15.32:0/3701151193 >> 172.21.15.32:0/3701151193 conn(0x7f91d409bc50 msgr2=0x7f91d40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- 172.21.15.32:0/3701151193 shutdown_connections 2023-10-16T08:17:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- 172.21.15.32:0/3701151193 wait complete. 2023-10-16T08:17:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 Processor -- start 2023-10-16T08:17:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- start start 2023-10-16T08:17:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d40d3010 0x7f91d40cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d415cb90 0x7f91d40cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b8250 con 0x7f91d40d15d0 2023-10-16T08:17:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b78b0 con 0x7f91d415cb90 2023-10-16T08:17:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dc7f8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b7b90 con 0x7f91d40d3010 2023-10-16T08:17:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91da594640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91da594640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d414f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50570/0 (socket says 172.21.15.32:50570) 2023-10-16T08:17:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91da594640 1 -- 172.21.15.32:0/1314330317 learned_addr learned my addr 172.21.15.32:0/1314330317 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91d9d93640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d40d3010 0x7f91d40cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.113+0000 7f91dad95640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d415cb90 0x7f91d40cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91da594640 1 -- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d40d3010 msgr2=0x7f91d40cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91da594640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d40d3010 0x7f91d40cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91da594640 1 -- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d415cb90 msgr2=0x7f91d40cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91da594640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d415cb90 0x7f91d40cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91da594640 1 -- 172.21.15.32:0/1314330317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc009a70 con 0x7f91d40d15d0 2023-10-16T08:17:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91da594640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d414f6b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f91d000d7d0 tx=0x7f91d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00059a0 con 0x7f91d40d15d0 2023-10-16T08:17:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d0005b00 con 0x7f91d40d15d0 2023-10-16T08:17:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40b7e70 con 0x7f91d40d15d0 2023-10-16T08:17:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0005e90 con 0x7f91d40d15d0 2023-10-16T08:17:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d414fbf0 con 0x7f91d40d15d0 2023-10-16T08:17:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d414fe70 con 0x7f91d40d15d0 2023-10-16T08:17:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.117+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f91d0004030 con 0x7f91d40d15d0 2023-10-16T08:17:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.121+0000 7f91caffd640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91b80664d0 0x7f91b8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.121+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f91d0083a30 con 0x7f91d40d15d0 2023-10-16T08:17:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.121+0000 7f91d9d93640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91b80664d0 0x7f91b8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.121+0000 7f91d9d93640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91b80664d0 0x7f91b8068990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91d40b6220 tx=0x7f91c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.125+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f91d0054c90 con 0x7f91d40d15d0 2023-10-16T08:17:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.337+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f91d4074010 con 0x7f91d40d15d0 2023-10-16T08:17:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.337+0000 7f91caffd640 1 -- 172.21.15.32:0/1314330317 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f91d0054820 con 0x7f91d40d15d0 2023-10-16T08:17:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91b80664d0 msgr2=0x7f91b8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91b80664d0 0x7f91b8068990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91d40b6220 tx=0x7f91c4009040 comp rx=0 tx=0).stop 2023-10-16T08:17:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 msgr2=0x7f91d414f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d414f6b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f91d000d7d0 tx=0x7f91d000dc90 comp rx=0 tx=0).stop 2023-10-16T08:17:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 shutdown_connections 2023-10-16T08:17:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d415cb90 0x7f91d40cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f91b80664d0 0x7f91b8068990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f91d40d3010 0x7f91d40cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 --2- 172.21.15.32:0/1314330317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d40d15d0 0x7f91d414f6b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 >> 172.21.15.32:0/1314330317 conn(0x7f91d409bc50 msgr2=0x7f91d40b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 shutdown_connections 2023-10-16T08:17:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.341+0000 7f91dc7f8640 1 -- 172.21.15.32:0/1314330317 wait complete. 2023-10-16T08:17:46.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-16T08:17:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2986695640 1 Processor -- start 2023-10-16T08:17:46.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2986695640 1 -- start start 2023-10-16T08:17:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2986695640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29800d1490 0x7f298006af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2986695640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800d3510 con 0x7f298015cd60 2023-10-16T08:17:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2986695640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800d2f50 con 0x7f298015c980 2023-10-16T08:17:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2986695640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800cd2b0 con 0x7f29800d1490 2023-10-16T08:17:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2985e94640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f298015cd60 0x7f29800cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59460/0 (socket says 172.21.15.32:59460) 2023-10-16T08:17:46.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2985e94640 1 -- 172.21.15.32:0/2013283765 learned_addr learned my addr 172.21.15.32:0/2013283765 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2984e92640 1 --2- 172.21.15.32:0/2013283765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29800d1490 0x7f298006af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888252114 0 0) 0x7f29780014a0 con 0x7f298015cd60 2023-10-16T08:17:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f296c003530 con 0x7f298015cd60 2023-10-16T08:17:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141913975 0 0) 0x7f29800d2f50 con 0x7f298015c980 2023-10-16T08:17:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29780014a0 con 0x7f298015c980 2023-10-16T08:17:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2984e92640 1 -- 172.21.15.32:0/2013283765 >> v1:172.21.15.186:6789/0 conn(0x7f298015c980 legacy=0x7f29800d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2984e92640 1 -- 172.21.15.32:0/2013283765 >> v1:172.21.15.32:6789/0 conn(0x7f298015cd60 legacy=0x7f29800cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.505+0000 7f2984e92640 1 -- 172.21.15.32:0/2013283765 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800d3230 con 0x7f29800d1490 2023-10-16T08:17:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2984e92640 1 --2- 172.21.15.32:0/2013283765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29800d1490 0x7f298006af90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29700097f0 tx=0x7f297001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8098ad4c4cd24b6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700aa640 con 0x7f29800d1490 2023-10-16T08:17:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29700aa7a0 con 0x7f29800d1490 2023-10-16T08:17:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2977fff640 1 -- 172.21.15.32:0/2013283765 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700aab30 con 0x7f29800d1490 2023-10-16T08:17:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 -- 172.21.15.32:0/2013283765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29800d1490 msgr2=0x7f298006af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 --2- 172.21.15.32:0/2013283765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29800d1490 0x7f298006af90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29700097f0 tx=0x7f297001c4c0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 -- 172.21.15.32:0/2013283765 shutdown_connections 2023-10-16T08:17:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 --2- 172.21.15.32:0/2013283765 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29800d1490 0x7f298006af90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 -- 172.21.15.32:0/2013283765 >> 172.21.15.32:0/2013283765 conn(0x7f298009ba80 msgr2=0x7f29800d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 -- 172.21.15.32:0/2013283765 shutdown_connections 2023-10-16T08:17:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.509+0000 7f2986695640 1 -- 172.21.15.32:0/2013283765 wait complete. 2023-10-16T08:17:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 Processor -- start 2023-10-16T08:17:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 -- start start 2023-10-16T08:17:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29800d1490 0x7f29800b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 0x7f29800c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f298015cd60 0x7f29800b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980156c50 con 0x7f298015c980 2023-10-16T08:17:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b7180 con 0x7f29800d1490 2023-10-16T08:17:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b7460 con 0x7f298015cd60 2023-10-16T08:17:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 0x7f29800c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 0x7f29800c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50576/0 (socket says 172.21.15.32:50576) 2023-10-16T08:17:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 -- 172.21.15.32:0/2983332879 learned_addr learned my addr 172.21.15.32:0/2983332879 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2985693640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29800d1490 0x7f29800b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2985e94640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f298015cd60 0x7f29800b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 -- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f298015cd60 msgr2=0x7f29800b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f298015cd60 0x7f29800b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 -- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29800d1490 msgr2=0x7f29800b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29800d1490 0x7f29800b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 -- 172.21.15.32:0/2983332879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2970009420 con 0x7f298015c980 2023-10-16T08:17:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2985e94640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f298015cd60 0x7f29800b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2984e92640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 0x7f29800c8130 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29700ac040 tx=0x7f297001ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700be410 con 0x7f298015c980 2023-10-16T08:17:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29700be590 con 0x7f298015c980 2023-10-16T08:17:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700be940 con 0x7f298015c980 2023-10-16T08:17:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.513+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29800b7740 con 0x7f298015c980 2023-10-16T08:17:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.517+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29800b2a50 con 0x7f298015c980 2023-10-16T08:17:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.517+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29800b53c0 con 0x7f298015c980 2023-10-16T08:17:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.517+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29700a8050 con 0x7f298015c980 2023-10-16T08:17:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.517+0000 7f2975ffb640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29640664a0 0x7f2964068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.517+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f297011de20 con 0x7f298015c980 2023-10-16T08:17:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.517+0000 7f2985693640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29640664a0 0x7f2964068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.521+0000 7f2985693640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29640664a0 0x7f2964068960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f297c001240 tx=0x7f297c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.521+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29800b53c0 con 0x7f298015c980 2023-10-16T08:17:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.733+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f29800b53c0 con 0x7f298015c980 2023-10-16T08:17:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.737+0000 7f2975ffb640 1 -- 172.21.15.32:0/2983332879 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f29800b53c0 con 0x7f298015c980 2023-10-16T08:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29640664a0 msgr2=0x7f2964068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29640664a0 0x7f2964068960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f297c001240 tx=0x7f297c009040 comp rx=0 tx=0).stop 2023-10-16T08:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 msgr2=0x7f29800c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 0x7f29800c8130 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29700ac040 tx=0x7f297001ad00 comp rx=0 tx=0).stop 2023-10-16T08:17:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 shutdown_connections 2023-10-16T08:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29640664a0 0x7f2964068960 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f298015cd60 0x7f29800b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f298015c980 0x7f29800c8130 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 --2- 172.21.15.32:0/2983332879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29800d1490 0x7f29800b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 >> 172.21.15.32:0/2983332879 conn(0x7f298009ba80 msgr2=0x7f298006be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 shutdown_connections 2023-10-16T08:17:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.741+0000 7f2986695640 1 -- 172.21.15.32:0/2983332879 wait complete. 2023-10-16T08:17:46.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-16T08:17:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 Processor -- start 2023-10-16T08:17:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 -- start start 2023-10-16T08:17:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc758065a80 0x7fc758161000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc75806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7580ccbd0 con 0x7fc75815cb30 2023-10-16T08:17:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75806b180 con 0x7fc758065a80 2023-10-16T08:17:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc75c935640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75806b460 con 0x7fc75815cf10 2023-10-16T08:17:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc758065a80 0x7fc758161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc758065a80 0x7fc758161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50584/0 (socket says 172.21.15.32:50584) 2023-10-16T08:17:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 -- 172.21.15.32:0/3019136992 learned_addr learned my addr 172.21.15.32:0/3019136992 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc756d9d640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc75806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 -- 172.21.15.32:0/3019136992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 msgr2=0x7fc75806cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc75806cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 -- 172.21.15.32:0/3019136992 >> v1:172.21.15.186:6789/0 conn(0x7fc75815cb30 legacy=0x7fc7581608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc74ffff640 1 -- 172.21.15.32:0/3019136992 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75806c6e0 con 0x7fc758065a80 2023-10-16T08:17:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.905+0000 7fc756d9d640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc75806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc74ffff640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc758161000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc740009870 tx=0x7fc74001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67393b8d20060fe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc755d9b640 1 -- 172.21.15.32:0/3019136992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400ab680 con 0x7fc758065a80 2023-10-16T08:17:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc755d9b640 1 -- 172.21.15.32:0/3019136992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7400ab7e0 con 0x7fc758065a80 2023-10-16T08:17:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc755d9b640 1 -- 172.21.15.32:0/3019136992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400abb70 con 0x7fc758065a80 2023-10-16T08:17:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 -- 172.21.15.32:0/3019136992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 msgr2=0x7fc758161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc758161000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc740009870 tx=0x7fc74001c480 comp rx=0 tx=0).stop 2023-10-16T08:17:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 -- 172.21.15.32:0/3019136992 shutdown_connections 2023-10-16T08:17:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc75806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 --2- 172.21.15.32:0/3019136992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc758161000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 -- 172.21.15.32:0/3019136992 >> 172.21.15.32:0/3019136992 conn(0x7fc7580b0880 msgr2=0x7fc7580b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 -- 172.21.15.32:0/3019136992 shutdown_connections 2023-10-16T08:17:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.909+0000 7fc75c935640 1 -- 172.21.15.32:0/3019136992 wait complete. 2023-10-16T08:17:46.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 Processor -- start 2023-10-16T08:17:46.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 -- start start 2023-10-16T08:17:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc7581597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc75815cb30 0x7fc7580bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc7580c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581f5140 con 0x7fc758065a80 2023-10-16T08:17:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758158130 con 0x7fc75815cb30 2023-10-16T08:17:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580c0a10 con 0x7fc75815cf10 2023-10-16T08:17:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc7581597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc7581597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50598/0 (socket says 172.21.15.32:50598) 2023-10-16T08:17:46.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 -- 172.21.15.32:0/27120457 learned_addr learned my addr 172.21.15.32:0/27120457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc756d9d640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc7580c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc74ffff640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc75815cb30 0x7fc7580bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 -- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 msgr2=0x7fc7580c14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc7580c14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 -- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc75815cb30 msgr2=0x7fc7580bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc75815cb30 0x7fc7580bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 -- 172.21.15.32:0/27120457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400094d0 con 0x7fc758065a80 2023-10-16T08:17:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc74ffff640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc75815cb30 0x7fc7580bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75659c640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc7581597b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c500 tx=0x7fc74800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748005220 con 0x7fc758065a80 2023-10-16T08:17:46.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748005380 con 0x7fc758065a80 2023-10-16T08:17:46.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7580c0cf0 con 0x7fc758065a80 2023-10-16T08:17:46.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748007270 con 0x7fc758065a80 2023-10-16T08:17:46.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.913+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7581554e0 con 0x7fc758065a80 2023-10-16T08:17:46.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.917+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc758074010 con 0x7fc758065a80 2023-10-16T08:17:46.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.917+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc7480073d0 con 0x7fc758065a80 2023-10-16T08:17:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.917+0000 7fc74effd640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc7380664a0 0x7fc738068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.917+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc748083670 con 0x7fc758065a80 2023-10-16T08:17:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.917+0000 7fc74ffff640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc7380664a0 0x7fc738068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:46.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.917+0000 7fc74ffff640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc7380664a0 0x7fc738068960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7400af000 tx=0x7fc7400a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:46.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:46.921+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc748050c70 con 0x7fc758065a80 2023-10-16T08:17:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.137+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc7580c11a0 con 0x7fc758065a80 2023-10-16T08:17:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc74effd640 1 -- 172.21.15.32:0/27120457 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc748054290 con 0x7fc758065a80 2023-10-16T08:17:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc7380664a0 msgr2=0x7fc738068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc7380664a0 0x7fc738068960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7400af000 tx=0x7fc7400a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 msgr2=0x7fc7581597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc7581597b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c500 tx=0x7fc74800c9c0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 shutdown_connections 2023-10-16T08:17:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc7380664a0 0x7fc738068960 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc75815cf10 0x7fc7580c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc75815cb30 0x7fc7580bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 --2- 172.21.15.32:0/27120457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc758065a80 0x7fc7581597b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 >> 172.21.15.32:0/27120457 conn(0x7fc7580b0880 msgr2=0x7fc758155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 shutdown_connections 2023-10-16T08:17:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.141+0000 7fc75c935640 1 -- 172.21.15.32:0/27120457 wait complete. 2023-10-16T08:17:47.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-16T08:17:47.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:17:47.216 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:17:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.305+0000 7fbaacdc4640 1 Processor -- start 2023-10-16T08:17:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.305+0000 7fbaacdc4640 1 -- start start 2023-10-16T08:17:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaacdc4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbaa815cb30 0x7fbaa81608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaacdc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaacdc4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa80ccbd0 con 0x7fbaa8065a80 2023-10-16T08:17:47.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaacdc4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa806b180 con 0x7fbaa815cb30 2023-10-16T08:17:47.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaacdc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa806b460 con 0x7fbaa815cf10 2023-10-16T08:17:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa6d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa659c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbaa815cb30 0x7fbaa81608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa5d9b640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7fbaa8065a80 0x7fbaa8160e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37772/0 (socket says 172.21.15.32:37772) 2023-10-16T08:17:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa5d9b640 1 -- 172.21.15.32:0/3642384777 learned_addr learned my addr 172.21.15.32:0/3642384777 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa559a640 1 -- 172.21.15.32:0/3642384777 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608920717 0 0) 0x7fbaa80ccbd0 con 0x7fbaa8065a80 2023-10-16T08:17:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa559a640 1 -- 172.21.15.32:0/3642384777 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba90003530 con 0x7fbaa8065a80 2023-10-16T08:17:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa6d9d640 1 -- 172.21.15.32:0/3642384777 >> v2:172.21.15.186:3300/0 conn(0x7fbaa815cb30 msgr2=0x7fbaa81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa6d9d640 1 --2- 172.21.15.32:0/3642384777 >> v2:172.21.15.186:3300/0 conn(0x7fbaa815cb30 0x7fbaa81608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa6d9d640 1 -- 172.21.15.32:0/3642384777 >> v1:172.21.15.186:6789/0 conn(0x7fbaa8065a80 legacy=0x7fbaa8160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.309+0000 7fbaa6d9d640 1 -- 172.21.15.32:0/3642384777 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa806c6e0 con 0x7fbaa815cf10 2023-10-16T08:17:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaa6d9d640 1 --2- 172.21.15.32:0/3642384777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa806cee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009d90 tx=0x7fbaa001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c66b81f65eee7d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaa559a640 1 -- 172.21.15.32:0/3642384777 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00a5950 con 0x7fbaa815cf10 2023-10-16T08:17:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaa559a640 1 -- 172.21.15.32:0/3642384777 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa00a5ab0 con 0x7fbaa815cf10 2023-10-16T08:17:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaa559a640 1 -- 172.21.15.32:0/3642384777 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00a5e40 con 0x7fbaa815cf10 2023-10-16T08:17:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3642384777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 msgr2=0x7fbaa806cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3642384777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa806cee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009d90 tx=0x7fbaa001c890 comp rx=0 tx=0).stop 2023-10-16T08:17:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3642384777 shutdown_connections 2023-10-16T08:17:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3642384777 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa806cee0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3642384777 >> v2:172.21.15.186:3300/0 conn(0x7fbaa815cb30 0x7fbaa81608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3642384777 >> 172.21.15.32:0/3642384777 conn(0x7fbaa80b0880 msgr2=0x7fbaa80b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3642384777 shutdown_connections 2023-10-16T08:17:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3642384777 wait complete. 2023-10-16T08:17:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 Processor -- start 2023-10-16T08:17:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.313+0000 7fbaacdc4640 1 -- start start 2023-10-16T08:17:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 0x7fbaa8156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaa815cb30 0x7fbaa80b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa809d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8156be0 con 0x7fbaa8065a80 2023-10-16T08:17:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8156ec0 con 0x7fbaa815cb30 2023-10-16T08:17:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa81571a0 con 0x7fbaa815cf10 2023-10-16T08:17:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 0x7fbaa8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 0x7fbaa8156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50608/0 (socket says 172.21.15.32:50608) 2023-10-16T08:17:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 -- 172.21.15.32:0/3541822063 learned_addr learned my addr 172.21.15.32:0/3541822063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa5d9b640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaa815cb30 0x7fbaa80b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa6d9d640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa809d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 -- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 msgr2=0x7fbaa809d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa809d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 -- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaa815cb30 msgr2=0x7fbaa80b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaa815cb30 0x7fbaa80b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 -- 172.21.15.32:0/3541822063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa0009a70 con 0x7fbaa8065a80 2023-10-16T08:17:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa6d9d640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa809d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaa659c640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 0x7fbaa8156390 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba9800e930 tx=0x7fba9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98009690 con 0x7fbaa8065a80 2023-10-16T08:17:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9800ca00 con 0x7fbaa8065a80 2023-10-16T08:17:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa8157480 con 0x7fbaa8065a80 2023-10-16T08:17:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9800cd70 con 0x7fbaa8065a80 2023-10-16T08:17:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa8157c80 con 0x7fbaa8065a80 2023-10-16T08:17:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa8074010 con 0x7fbaa8065a80 2023-10-16T08:17:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.317+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba98013470 con 0x7fbaa8065a80 2023-10-16T08:17:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.321+0000 7fba96ffd640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba880664d0 0x7fba88068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.321+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fba98083f40 con 0x7fbaa8065a80 2023-10-16T08:17:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.321+0000 7fbaa5d9b640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba880664d0 0x7fba88068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.321+0000 7fbaa5d9b640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba880664d0 0x7fba88068990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8067840 tx=0x7fba9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.325+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fba980515f0 con 0x7fbaa8065a80 2023-10-16T08:17:47.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.537+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fbaa80b4d10 con 0x7fbaa8065a80 2023-10-16T08:17:47.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fba96ffd640 1 -- 172.21.15.32:0/3541822063 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fba980170a0 con 0x7fbaa8065a80 2023-10-16T08:17:47.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba880664d0 msgr2=0x7fba88068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba880664d0 0x7fba88068990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8067840 tx=0x7fba9c009040 comp rx=0 tx=0).stop 2023-10-16T08:17:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 msgr2=0x7fbaa8156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 0x7fbaa8156390 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba9800e930 tx=0x7fba9800edf0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 shutdown_connections 2023-10-16T08:17:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fba880664d0 0x7fba88068990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbaa815cf10 0x7fbaa809d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaa815cb30 0x7fbaa80b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 --2- 172.21.15.32:0/3541822063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa8065a80 0x7fbaa8156390 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 >> 172.21.15.32:0/3541822063 conn(0x7fbaa80b0880 msgr2=0x7fbaa8070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 shutdown_connections 2023-10-16T08:17:47.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.541+0000 7fbaacdc4640 1 -- 172.21.15.32:0/3541822063 wait complete. 2023-10-16T08:17:47.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-16T08:17:47.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 Processor -- start 2023-10-16T08:17:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 -- start start 2023-10-16T08:17:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 0x7fd77c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd77c0d3010 0x7fd77c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c0ce770 con 0x7fd77c15cb90 2023-10-16T08:17:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0d0600 con 0x7fd77c0d3010 2023-10-16T08:17:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd783365640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0c6450 con 0x7fd77c0d15d0 2023-10-16T08:17:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 0x7fd77c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd781902640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd77c0d3010 0x7fd77c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 0x7fd77c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:41676/0 (socket says 172.21.15.32:41676) 2023-10-16T08:17:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 -- 172.21.15.32:0/1585984038 learned_addr learned my addr 172.21.15.32:0/1585984038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 -- 172.21.15.32:0/1585984038 >> v2:172.21.15.186:3300/0 conn(0x7fd77c0d3010 msgr2=0x7fd77c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 --2- 172.21.15.32:0/1585984038 >> v2:172.21.15.186:3300/0 conn(0x7fd77c0d3010 0x7fd77c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 -- 172.21.15.32:0/1585984038 >> v1:172.21.15.186:6789/0 conn(0x7fd77c15cb90 legacy=0x7fd77c0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd780900640 1 -- 172.21.15.32:0/1585984038 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0d08a0 con 0x7fd77c0d15d0 2023-10-16T08:17:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.709+0000 7fd781902640 1 --2- 172.21.15.32:0/1585984038 >> v2:172.21.15.186:3300/0 conn(0x7fd77c0d3010 0x7fd77c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd780900640 1 --2- 172.21.15.32:0/1585984038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 0x7fd77c0cfb10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009870 tx=0x7fd76c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d86c4f21fbe7c8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd773fff640 1 -- 172.21.15.32:0/1585984038 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0ab680 con 0x7fd77c0d15d0 2023-10-16T08:17:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd773fff640 1 -- 172.21.15.32:0/1585984038 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c0ab7e0 con 0x7fd77c0d15d0 2023-10-16T08:17:47.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd773fff640 1 -- 172.21.15.32:0/1585984038 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0abb70 con 0x7fd77c0d15d0 2023-10-16T08:17:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 -- 172.21.15.32:0/1585984038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 msgr2=0x7fd77c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 --2- 172.21.15.32:0/1585984038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 0x7fd77c0cfb10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009870 tx=0x7fd76c01c480 comp rx=0 tx=0).stop 2023-10-16T08:17:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 -- 172.21.15.32:0/1585984038 shutdown_connections 2023-10-16T08:17:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 --2- 172.21.15.32:0/1585984038 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c0d15d0 0x7fd77c0cfb10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 --2- 172.21.15.32:0/1585984038 >> v2:172.21.15.186:3300/0 conn(0x7fd77c0d3010 0x7fd77c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 -- 172.21.15.32:0/1585984038 >> 172.21.15.32:0/1585984038 conn(0x7fd77c09bc50 msgr2=0x7fd77c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 -- 172.21.15.32:0/1585984038 shutdown_connections 2023-10-16T08:17:47.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.713+0000 7fd783365640 1 -- 172.21.15.32:0/1585984038 wait complete. 2023-10-16T08:17:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 Processor -- start 2023-10-16T08:17:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 -- start start 2023-10-16T08:17:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd77c0d15d0 0x7fd77c153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 0x7fd77c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c15cb90 0x7fd77c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c152c20 con 0x7fd77c0d3010 2023-10-16T08:17:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c152310 con 0x7fd77c0d15d0 2023-10-16T08:17:47.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0cc8a0 con 0x7fd77c15cb90 2023-10-16T08:17:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 0x7fd77c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 0x7fd77c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50616/0 (socket says 172.21.15.32:50616) 2023-10-16T08:17:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 -- 172.21.15.32:0/3163493817 learned_addr learned my addr 172.21.15.32:0/3163493817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd781902640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c15cb90 0x7fd77c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd781101640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd77c0d15d0 0x7fd77c153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 -- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c15cb90 msgr2=0x7fd77c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c15cb90 0x7fd77c0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 -- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd77c0d15d0 msgr2=0x7fd77c153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd77c0d15d0 0x7fd77c153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 -- 172.21.15.32:0/3163493817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774009a70 con 0x7fd77c0d3010 2023-10-16T08:17:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd781101640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd77c0d15d0 0x7fd77c153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd780900640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 0x7fd77c0c9d00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0ad040 tx=0x7fd76c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0ab680 con 0x7fd77c0d3010 2023-10-16T08:17:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c0b4800 con 0x7fd77c0d3010 2023-10-16T08:17:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0b4bb0 con 0x7fd77c0d3010 2023-10-16T08:17:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c0094d0 con 0x7fd77c0d3010 2023-10-16T08:17:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.717+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c152840 con 0x7fd77c0d3010 2023-10-16T08:17:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.721+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c152d60 con 0x7fd77c0d3010 2023-10-16T08:17:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.721+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd76c0b4d10 con 0x7fd77c0d3010 2023-10-16T08:17:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.721+0000 7fd771ffb640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7600664a0 0x7fd760068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.721+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd76c11e670 con 0x7fd77c0d3010 2023-10-16T08:17:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.721+0000 7fd781101640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7600664a0 0x7fd760068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.721+0000 7fd781101640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7600664a0 0x7fd760068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0c5340 tx=0x7fd778009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.725+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd77c152d60 con 0x7fd77c0d3010 2023-10-16T08:17:47.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.941+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd77c152d60 con 0x7fd77c0d3010 2023-10-16T08:17:47.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.941+0000 7fd771ffb640 1 -- 172.21.15.32:0/3163493817 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd77c152d60 con 0x7fd77c0d3010 2023-10-16T08:17:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7600664a0 msgr2=0x7fd760068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7600664a0 0x7fd760068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0c5340 tx=0x7fd778009040 comp rx=0 tx=0).stop 2023-10-16T08:17:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 msgr2=0x7fd77c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 0x7fd77c0c9d00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0ad040 tx=0x7fd76c01acc0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 shutdown_connections 2023-10-16T08:17:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd7600664a0 0x7fd760068960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd77c15cb90 0x7fd77c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd77c0d3010 0x7fd77c0c9d00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 --2- 172.21.15.32:0/3163493817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd77c0d15d0 0x7fd77c153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 >> 172.21.15.32:0/3163493817 conn(0x7fd77c09bc50 msgr2=0x7fd77c06b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 shutdown_connections 2023-10-16T08:17:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:47.945+0000 7fd783365640 1 -- 172.21.15.32:0/3163493817 wait complete. 2023-10-16T08:17:47.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 Processor -- start 2023-10-16T08:17:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 -- start start 2023-10-16T08:17:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23cc0d15d0 0x7f23cc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d3010 0x7f23cc0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0ce770 con 0x7f23cc15cb90 2023-10-16T08:17:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0d0600 con 0x7f23cc0d15d0 2023-10-16T08:17:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d2c46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0c6450 con 0x7f23cc0d3010 2023-10-16T08:17:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23cc0d15d0 0x7f23cc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23cc0d15d0 0x7f23cc0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50622/0 (socket says 172.21.15.32:50622) 2023-10-16T08:17:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 -- 172.21.15.32:0/3569159343 learned_addr learned my addr 172.21.15.32:0/3569159343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23d11e3640 1 --2- 172.21.15.32:0/3569159343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d3010 0x7f23cc0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 -- 172.21.15.32:0/3569159343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d3010 msgr2=0x7f23cc0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 --2- 172.21.15.32:0/3569159343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d3010 0x7f23cc0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 -- 172.21.15.32:0/3569159343 >> v1:172.21.15.186:6789/0 conn(0x7f23cc15cb90 legacy=0x7f23cc0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.109+0000 7f23cbfff640 1 -- 172.21.15.32:0/3569159343 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0d08a0 con 0x7f23cc0d15d0 2023-10-16T08:17:48.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23cbfff640 1 --2- 172.21.15.32:0/3569159343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc0d15d0 0x7f23cc0cfb10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009870 tx=0x7f23b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833669ccc5e08cbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23cb7fe640 1 -- 172.21.15.32:0/3569159343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b80ab680 con 0x7f23cc0d15d0 2023-10-16T08:17:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23cb7fe640 1 -- 172.21.15.32:0/3569159343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b80ab7e0 con 0x7f23cc0d15d0 2023-10-16T08:17:48.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23cb7fe640 1 -- 172.21.15.32:0/3569159343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b80abb70 con 0x7f23cc0d15d0 2023-10-16T08:17:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 -- 172.21.15.32:0/3569159343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc0d15d0 msgr2=0x7f23cc0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 --2- 172.21.15.32:0/3569159343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc0d15d0 0x7f23cc0cfb10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009870 tx=0x7f23b801c480 comp rx=0 tx=0).stop 2023-10-16T08:17:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 -- 172.21.15.32:0/3569159343 shutdown_connections 2023-10-16T08:17:48.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 --2- 172.21.15.32:0/3569159343 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d3010 0x7f23cc0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 --2- 172.21.15.32:0/3569159343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc0d15d0 0x7f23cc0cfb10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 -- 172.21.15.32:0/3569159343 >> 172.21.15.32:0/3569159343 conn(0x7f23cc09bc50 msgr2=0x7f23cc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 -- 172.21.15.32:0/3569159343 shutdown_connections 2023-10-16T08:17:48.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.113+0000 7f23d2c46640 1 -- 172.21.15.32:0/3569159343 wait complete. 2023-10-16T08:17:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 Processor -- start 2023-10-16T08:17:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 -- start start 2023-10-16T08:17:48.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d15d0 0x7f23cc0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3010 0x7f23cc09ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 0x7f23cc09f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0a07d0 con 0x7f23cc15cb90 2023-10-16T08:17:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc09f8c0 con 0x7f23cc0d3010 2023-10-16T08:17:48.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d2c46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc09fb50 con 0x7f23cc0d15d0 2023-10-16T08:17:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 0x7f23cc09f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 0x7f23cc09f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50626/0 (socket says 172.21.15.32:50626) 2023-10-16T08:17:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 -- 172.21.15.32:0/4256938948 learned_addr learned my addr 172.21.15.32:0/4256938948 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d09e2640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d15d0 0x7f23cc0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23cbfff640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3010 0x7f23cc09ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 -- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d15d0 msgr2=0x7f23cc0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d15d0 0x7f23cc0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 -- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3010 msgr2=0x7f23cc09ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3010 0x7f23cc09ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 -- 172.21.15.32:0/4256938948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80094d0 con 0x7f23cc15cb90 2023-10-16T08:17:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d09e2640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d15d0 0x7f23cc0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.117+0000 7f23d11e3640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 0x7f23cc09f380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f23c4007e40 tx=0x7f23c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4014620 con 0x7f23cc15cb90 2023-10-16T08:17:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc09fe30 con 0x7f23cc15cb90 2023-10-16T08:17:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c400ee90 con 0x7f23cc15cb90 2023-10-16T08:17:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4014980 con 0x7f23cc15cb90 2023-10-16T08:17:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc0a03f0 con 0x7f23cc15cb90 2023-10-16T08:17:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc167550 con 0x7f23cc15cb90 2023-10-16T08:17:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.121+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23c4014ae0 con 0x7f23cc15cb90 2023-10-16T08:17:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.125+0000 7f23c97fa640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23b00664d0 0x7f23b0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.125+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f23c40845b0 con 0x7f23cc15cb90 2023-10-16T08:17:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.125+0000 7f23d09e2640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23b00664d0 0x7f23b0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.125+0000 7f23d09e2640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23b00664d0 0x7f23b0068990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23c00016c0 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.129+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23c4051ba0 con 0x7f23cc15cb90 2023-10-16T08:17:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.285+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23cc074010 con 0x7f23cc15cb90 2023-10-16T08:17:48.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.285+0000 7f23c97fa640 1 -- 172.21.15.32:0/4256938948 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13062 (secure 0 0 0) 0x7f23c40551c0 con 0x7f23cc15cb90 2023-10-16T08:17:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:48.294 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:45.122607+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:48.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23b00664d0 msgr2=0x7f23b0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23b00664d0 0x7f23b0068990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23c00016c0 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2023-10-16T08:17:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 msgr2=0x7f23cc09f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 0x7f23cc09f380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f23c4007e40 tx=0x7f23c400e4b0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 shutdown_connections 2023-10-16T08:17:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23cc15cb90 0x7f23cc09f380 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc0d3010 0x7f23cc09ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23b00664d0 0x7f23b0068990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 --2- 172.21.15.32:0/4256938948 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23cc0d15d0 0x7f23cc0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 >> 172.21.15.32:0/4256938948 conn(0x7f23cc09bc50 msgr2=0x7f23cc06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 shutdown_connections 2023-10-16T08:17:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.289+0000 7f23d2c46640 1 -- 172.21.15.32:0/4256938948 wait complete. 2023-10-16T08:17:48.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:17:48.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:17:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 Processor -- start 2023-10-16T08:17:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 -- start start 2023-10-16T08:17:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f86380d0210 0x7f86380d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86380b84e0 0x7f86380b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86380d1670 con 0x7f863815cba0 2023-10-16T08:17:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86380d1460 con 0x7f86380b84e0 2023-10-16T08:17:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863f617640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86380d1950 con 0x7f86380d0210 2023-10-16T08:17:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86380b84e0 0x7f86380b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86380b84e0 0x7f86380b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50638/0 (socket says 172.21.15.32:50638) 2023-10-16T08:17:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 -- 172.21.15.32:0/3122344270 learned_addr learned my addr 172.21.15.32:0/3122344270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863cbb2640 1 --2- 172.21.15.32:0/3122344270 >> v2:172.21.15.186:3300/0 conn(0x7f86380d0210 0x7f86380d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 -- 172.21.15.32:0/3122344270 >> v2:172.21.15.186:3300/0 conn(0x7f86380d0210 msgr2=0x7f86380d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 --2- 172.21.15.32:0/3122344270 >> v2:172.21.15.186:3300/0 conn(0x7f86380d0210 0x7f86380d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 -- 172.21.15.32:0/3122344270 >> v1:172.21.15.186:6789/0 conn(0x7f863815cba0 legacy=0x7f86380cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.457+0000 7f863dbb4640 1 -- 172.21.15.32:0/3122344270 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380d1c30 con 0x7f86380b84e0 2023-10-16T08:17:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863dbb4640 1 --2- 172.21.15.32:0/3122344270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b5cf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f86380cf010 tx=0x7f863001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a8e53b3a2177a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f862ffff640 1 -- 172.21.15.32:0/3122344270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300a5800 con 0x7f86380b84e0 2023-10-16T08:17:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f862ffff640 1 -- 172.21.15.32:0/3122344270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86300a5960 con 0x7f86380b84e0 2023-10-16T08:17:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f862ffff640 1 -- 172.21.15.32:0/3122344270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300a5cf0 con 0x7f86380b84e0 2023-10-16T08:17:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 -- 172.21.15.32:0/3122344270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 msgr2=0x7f86380b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 --2- 172.21.15.32:0/3122344270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b5cf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f86380cf010 tx=0x7f863001c940 comp rx=0 tx=0).stop 2023-10-16T08:17:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 -- 172.21.15.32:0/3122344270 shutdown_connections 2023-10-16T08:17:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 --2- 172.21.15.32:0/3122344270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b5cf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 --2- 172.21.15.32:0/3122344270 >> v2:172.21.15.186:3300/0 conn(0x7f86380d0210 0x7f86380d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 -- 172.21.15.32:0/3122344270 >> 172.21.15.32:0/3122344270 conn(0x7f86380719c0 msgr2=0x7f8638071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 -- 172.21.15.32:0/3122344270 shutdown_connections 2023-10-16T08:17:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.461+0000 7f863f617640 1 -- 172.21.15.32:0/3122344270 wait complete. 2023-10-16T08:17:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 Processor -- start 2023-10-16T08:17:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 -- start start 2023-10-16T08:17:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86380d0210 0x7f8638145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f863815cba0 0x7f86381f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86381463c0 con 0x7f86380b84e0 2023-10-16T08:17:48.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86380c5e50 con 0x7f86380d0210 2023-10-16T08:17:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86381f94a0 con 0x7f863815cba0 2023-10-16T08:17:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50648/0 (socket says 172.21.15.32:50648) 2023-10-16T08:17:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 -- 172.21.15.32:0/1869062082 learned_addr learned my addr 172.21.15.32:0/1869062082 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863cbb2640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86380d0210 0x7f8638145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863dbb4640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f863815cba0 0x7f86381f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 -- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f863815cba0 msgr2=0x7f86381f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f863815cba0 0x7f86381f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 -- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86380d0210 msgr2=0x7f8638145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86380d0210 0x7f8638145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 -- 172.21.15.32:0/1869062082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009b30 con 0x7f86380b84e0 2023-10-16T08:17:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863d3b3640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b6f50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f863400c3f0 tx=0x7f863400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340052d0 con 0x7f86380b84e0 2023-10-16T08:17:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634005430 con 0x7f86380b84e0 2023-10-16T08:17:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634003840 con 0x7f86380b84e0 2023-10-16T08:17:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86380cb270 con 0x7f86380b84e0 2023-10-16T08:17:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.465+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86380cb830 con 0x7f86380b84e0 2023-10-16T08:17:48.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.469+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86380cd190 con 0x7f86380b84e0 2023-10-16T08:17:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.469+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86340039a0 con 0x7f86380b84e0 2023-10-16T08:17:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.469+0000 7f862dffb640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f861c0664a0 0x7f861c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.469+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8634083070 con 0x7f86380b84e0 2023-10-16T08:17:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.469+0000 7f863cbb2640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f861c0664a0 0x7f861c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.469+0000 7f863cbb2640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f861c0664a0 0x7f861c068960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8638155930 tx=0x7f8628009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.473+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8634050840 con 0x7f86380b84e0 2023-10-16T08:17:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.633+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f863804e170 con 0x7f86380b84e0 2023-10-16T08:17:48.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.633+0000 7f862dffb640 1 -- 172.21.15.32:0/1869062082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8634017020 con 0x7f86380b84e0 2023-10-16T08:17:48.640 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:17:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.633+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f861c0664a0 msgr2=0x7f861c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.633+0000 7f863f617640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f861c0664a0 0x7f861c068960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8638155930 tx=0x7f8628009190 comp rx=0 tx=0).stop 2023-10-16T08:17:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.633+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 msgr2=0x7f86380b6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.633+0000 7f863f617640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b6f50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f863400c3f0 tx=0x7f863400c8b0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 shutdown_connections 2023-10-16T08:17:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f861c0664a0 0x7f861c068960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f863815cba0 0x7f86381f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86380d0210 0x7f8638145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 --2- 172.21.15.32:0/1869062082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86380b84e0 0x7f86380b6f50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 >> 172.21.15.32:0/1869062082 conn(0x7f86380719c0 msgr2=0x7f86381541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 shutdown_connections 2023-10-16T08:17:48.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.637+0000 7f863f617640 1 -- 172.21.15.32:0/1869062082 wait complete. 2023-10-16T08:17:48.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T08:17:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40c006b640 1 Processor -- start 2023-10-16T08:17:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40c006b640 1 -- start start 2023-10-16T08:17:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40c006b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40b80d15d0 0x7f40b80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40c006b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b80ce770 con 0x7f40b815cb90 2023-10-16T08:17:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40c006b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b80d0600 con 0x7f40b80d3010 2023-10-16T08:17:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40c006b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b80c6450 con 0x7f40b80d15d0 2023-10-16T08:17:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bd606640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40b80d15d0 0x7f40b80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bde07640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40b815cb90 0x7f40b80d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59464/0 (socket says 172.21.15.32:59464) 2023-10-16T08:17:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bde07640 1 -- 172.21.15.32:0/2624472295 learned_addr learned my addr 172.21.15.32:0/2624472295 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:48.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bd606640 1 -- 172.21.15.32:0/2624472295 >> v1:172.21.15.186:6789/0 conn(0x7f40b80d3010 legacy=0x7f40b80c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bd606640 1 -- 172.21.15.32:0/2624472295 >> v1:172.21.15.32:6789/0 conn(0x7f40b815cb90 legacy=0x7f40b80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bd606640 1 -- 172.21.15.32:0/2624472295 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80d08a0 con 0x7f40b80d15d0 2023-10-16T08:17:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.801+0000 7f40bce05640 1 -- 172.21.15.32:0/2624472295 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708890178 0 0) 0x7f40b80ce770 con 0x7f40b815cb90 2023-10-16T08:17:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40bd606640 1 --2- 172.21.15.32:0/2624472295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b80cfb10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f40a80097c0 tx=0x7f40a801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b26314c6445c9b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40bce05640 1 -- 172.21.15.32:0/2624472295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80ac620 con 0x7f40b80d15d0 2023-10-16T08:17:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40bce05640 1 -- 172.21.15.32:0/2624472295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a80ac780 con 0x7f40b80d15d0 2023-10-16T08:17:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40bce05640 1 -- 172.21.15.32:0/2624472295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80acb10 con 0x7f40b80d15d0 2023-10-16T08:17:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 -- 172.21.15.32:0/2624472295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 msgr2=0x7f40b80cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 --2- 172.21.15.32:0/2624472295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b80cfb10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f40a80097c0 tx=0x7f40a801c3c0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 -- 172.21.15.32:0/2624472295 shutdown_connections 2023-10-16T08:17:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 --2- 172.21.15.32:0/2624472295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b80cfb10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 -- 172.21.15.32:0/2624472295 >> 172.21.15.32:0/2624472295 conn(0x7f40b809bc50 msgr2=0x7f40b80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 -- 172.21.15.32:0/2624472295 shutdown_connections 2023-10-16T08:17:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.805+0000 7f40c006b640 1 -- 172.21.15.32:0/2624472295 wait complete. 2023-10-16T08:17:48.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 Processor -- start 2023-10-16T08:17:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- start start 2023-10-16T08:17:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b80d3010 0x7f40b814d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40b815cb90 0x7f40b81547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b814e4a0 con 0x7f40b80d15d0 2023-10-16T08:17:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8153070 con 0x7f40b80d3010 2023-10-16T08:17:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8153350 con 0x7f40b815cb90 2023-10-16T08:17:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b81571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50664/0 (socket says 172.21.15.32:50664) 2023-10-16T08:17:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 -- 172.21.15.32:0/3268000591 learned_addr learned my addr 172.21.15.32:0/3268000591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bd606640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b80d3010 0x7f40b814d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40be608640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40b815cb90 0x7f40b81547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 -- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40b815cb90 msgr2=0x7f40b81547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40b815cb90 0x7f40b81547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 -- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b80d3010 msgr2=0x7f40b814d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b80d3010 0x7f40b814d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 -- 172.21.15.32:0/3268000591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8009420 con 0x7f40b80d15d0 2023-10-16T08:17:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40bde07640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b81571e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c3f0 tx=0x7f40b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40074e0 con 0x7f40b80d15d0 2023-10-16T08:17:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4007640 con 0x7f40b80d15d0 2023-10-16T08:17:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40059f0 con 0x7f40b80d15d0 2023-10-16T08:17:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b8157840 con 0x7f40b80d15d0 2023-10-16T08:17:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b8157e00 con 0x7f40b80d15d0 2023-10-16T08:17:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.809+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b8074010 con 0x7f40b80d15d0 2023-10-16T08:17:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.813+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f40b4005b50 con 0x7f40b80d15d0 2023-10-16T08:17:48.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.813+0000 7f40ae7fc640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f409c0664a0 0x7f409c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:48.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.813+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f40b4083260 con 0x7f40b80d15d0 2023-10-16T08:17:48.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.813+0000 7f40bd606640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f409c0664a0 0x7f409c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:48.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.813+0000 7f40bd606640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f409c0664a0 0x7f409c068960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40a80ae000 tx=0x7f40a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:48.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.817+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f40b4050900 con 0x7f40b80d15d0 2023-10-16T08:17:48.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.977+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f40b8157ff0 con 0x7f40b80d15d0 2023-10-16T08:17:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40ae7fc640 1 -- 172.21.15.32:0/3268000591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f40b4017060 con 0x7f40b80d15d0 2023-10-16T08:17:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-10-16T08:17:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:17:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-10-16T08:17:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-10-16T08:17:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-10-16T08:17:48.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f409c0664a0 msgr2=0x7f409c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f409c0664a0 0x7f409c068960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40a80ae000 tx=0x7f40a8008040 comp rx=0 tx=0).stop 2023-10-16T08:17:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 msgr2=0x7f40b81571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b81571e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c3f0 tx=0x7f40b400c8b0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 shutdown_connections 2023-10-16T08:17:48.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f409c0664a0 0x7f409c068960 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f40b815cb90 0x7f40b81547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b80d3010 0x7f40b814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 --2- 172.21.15.32:0/3268000591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b80d15d0 0x7f40b81571e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 >> 172.21.15.32:0/3268000591 conn(0x7f40b809bc50 msgr2=0x7f40b806c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:48.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 shutdown_connections 2023-10-16T08:17:48.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:48.981+0000 7f40c006b640 1 -- 172.21.15.32:0/3268000591 wait complete. 2023-10-16T08:17:49.007 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'testcephfs1' 2023-10-16T08:17:49.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_metadata --pg_num_min 64 2023-10-16T08:17:49.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34e454640 1 Processor -- start 2023-10-16T08:17:49.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34e454640 1 -- start start 2023-10-16T08:17:49.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34e454640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd3480cfd20 0x7fd3480d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34e454640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3480ce460 con 0x7fd3480cdc80 2023-10-16T08:17:49.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34e454640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34815da30 con 0x7fd3480d2570 2023-10-16T08:17:49.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34e454640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3480cd830 con 0x7fd3480cfd20 2023-10-16T08:17:49.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd3477fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3480cdc80 0x7fd3480cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59480/0 (socket says 172.21.15.32:59480) 2023-10-16T08:17:49.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd3477fe640 1 -- 172.21.15.32:0/926198664 learned_addr learned my addr 172.21.15.32:0/926198664 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:49.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd34c9f1640 1 --2- 172.21.15.32:0/926198664 >> v2:172.21.15.186:3300/0 conn(0x7fd3480cfd20 0x7fd3480d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643596264 0 0) 0x7fd3480ce460 con 0x7fd3480cdc80 2023-10-16T08:17:49.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd330003530 con 0x7fd3480cdc80 2023-10-16T08:17:49.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 188979829 0 0) 0x7fd330003530 con 0x7fd3480cdc80 2023-10-16T08:17:49.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 >> v2:172.21.15.186:3300/0 conn(0x7fd3480cfd20 msgr2=0x7fd3480d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:49.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd346ffd640 1 --2- 172.21.15.32:0/926198664 >> v2:172.21.15.186:3300/0 conn(0x7fd3480cfd20 0x7fd3480d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.149+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 >> v1:172.21.15.186:6789/0 conn(0x7fd3480d2570 legacy=0x7fd348161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:49.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480ce180 con 0x7fd3480cdc80 2023-10-16T08:17:49.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fd33c001680 con 0x7fd3480cdc80 2023-10-16T08:17:49.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 >> v1:172.21.15.32:6789/0 conn(0x7fd3480cdc80 legacy=0x7fd3480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:49.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330003980 0x7fd330003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3300047d0 0x7fd330004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330005190 0x7fd33000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480cd830 con 0x7fd330003980 2023-10-16T08:17:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000c030 con 0x7fd3300047d0 2023-10-16T08:17:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000c1f0 con 0x7fd330005190 2023-10-16T08:17:49.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330003980 0x7fd330003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd3477fe640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3300047d0 0x7fd330004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd34c9f1640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330005190 0x7fd33000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 -- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330005190 msgr2=0x7fd33000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330005190 0x7fd33000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 -- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3300047d0 msgr2=0x7fd330004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3300047d0 0x7fd330004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 -- 172.21.15.32:0/926198664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd330003530 con 0x7fd330003980 2023-10-16T08:17:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd34c9f1640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330005190 0x7fd33000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:49.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd34c9f1640 1 -- 172.21.15.32:0/926198664 reap_dead start 2023-10-16T08:17:49.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd347fff640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330003980 0x7fd330003da0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd338001780 tx=0x7fd33801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c035b20309b0bd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3380052d0 con 0x7fd330003980 2023-10-16T08:17:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3380a7050 con 0x7fd330003980 2023-10-16T08:17:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.153+0000 7fd346ffd640 1 -- 172.21.15.32:0/926198664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338003600 con 0x7fd330003980 2023-10-16T08:17:49.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- 172.21.15.32:0/926198664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330003980 msgr2=0x7fd330003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330003980 0x7fd330003da0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd338001780 tx=0x7fd33801c3b0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- 172.21.15.32:0/926198664 shutdown_connections 2023-10-16T08:17:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 --2- 172.21.15.32:0/926198664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330003980 0x7fd330003da0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- 172.21.15.32:0/926198664 >> 172.21.15.32:0/926198664 conn(0x7fd3480715c0 msgr2=0x7fd3480719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- 172.21.15.32:0/926198664 shutdown_connections 2023-10-16T08:17:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- 172.21.15.32:0/926198664 wait complete. 2023-10-16T08:17:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 Processor -- start 2023-10-16T08:17:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- start start 2023-10-16T08:17:49.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330003980 0x7fd3480cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 0x7fd3480cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3480d1670 0x7fd3480d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480d2030 con 0x7fd3480c8fc0 2023-10-16T08:17:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480ccfe0 con 0x7fd3480d1670 2023-10-16T08:17:49.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd34e454640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480ccd00 con 0x7fd330003980 2023-10-16T08:17:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd3477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 0x7fd3480cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd3477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 0x7fd3480cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50690/0 (socket says 172.21.15.32:50690) 2023-10-16T08:17:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.157+0000 7fd3477fe640 1 -- 172.21.15.32:0/2422106921 learned_addr learned my addr 172.21.15.32:0/2422106921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd347fff640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330003980 0x7fd3480cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd34c9f1640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3480d1670 0x7fd3480d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd3477fe640 1 -- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330003980 msgr2=0x7fd3480cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd3477fe640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330003980 0x7fd3480cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd3477fe640 1 -- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3480d1670 msgr2=0x7fd3480d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd3477fe640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3480d1670 0x7fd3480d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:49.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd3477fe640 1 -- 172.21.15.32:0/2422106921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480c67f0 con 0x7fd3480c8fc0 2023-10-16T08:17:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd3477fe640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 0x7fd3480cc370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd33c007290 tx=0x7fd33c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00f9c0 con 0x7fd3480c8fc0 2023-10-16T08:17:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c00fb20 con 0x7fd3480c8fc0 2023-10-16T08:17:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3480c6ad0 con 0x7fd3480c8fc0 2023-10-16T08:17:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00fed0 con 0x7fd3480c8fc0 2023-10-16T08:17:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3480c9440 con 0x7fd3480c8fc0 2023-10-16T08:17:49.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34804e170 con 0x7fd3480c8fc0 2023-10-16T08:17:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd33c01d460 con 0x7fd3480c8fc0 2023-10-16T08:17:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.161+0000 7fd344ff9640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3340664d0 0x7fd334068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.165+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd33c083f00 con 0x7fd3480c8fc0 2023-10-16T08:17:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.165+0000 7fd347fff640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3340664d0 0x7fd334068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.165+0000 7fd347fff640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3340664d0 0x7fd334068990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd338001750 tx=0x7fd3380011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.169+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd33c051590 con 0x7fd3480c8fc0 2023-10-16T08:17:49.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:49.325+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd3480c6f80 con 0x7fd3480c8fc0 2023-10-16T08:17:50.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.125+0000 7fd344ff9640 1 -- 172.21.15.32:0/2422106921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64}]=0 pool 'testcephfs1_metadata' created v36) v1 ==== 148+0+0 (secure 0 0 0) 0x7fd33c054bb0 con 0x7fd3480c8fc0 2023-10-16T08:17:50.133 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_metadata' created 2023-10-16T08:17:50.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3340664d0 msgr2=0x7fd334068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3340664d0 0x7fd334068990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd338001750 tx=0x7fd3380011a0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 msgr2=0x7fd3480cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 0x7fd3480cc370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd33c007290 tx=0x7fd33c007750 comp rx=0 tx=0).stop 2023-10-16T08:17:50.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 shutdown_connections 2023-10-16T08:17:50.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3480d1670 0x7fd3480d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480c8fc0 0x7fd3480cc370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd3340664d0 0x7fd334068990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 --2- 172.21.15.32:0/2422106921 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd330003980 0x7fd3480cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.129+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 >> 172.21.15.32:0/2422106921 conn(0x7fd3480715c0 msgr2=0x7fd34815fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:50.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.133+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 shutdown_connections 2023-10-16T08:17:50.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.133+0000 7fd34e454640 1 -- 172.21.15.32:0/2422106921 wait complete. 2023-10-16T08:17:50.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T08:17:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.305+0000 7f3486b1d640 1 Processor -- start 2023-10-16T08:17:50.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.305+0000 7f3486b1d640 1 -- start start 2023-10-16T08:17:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.305+0000 7f3486b1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f348015cdd0 0x7f3480069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:50.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.305+0000 7f3486b1d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3480160c40 con 0x7f3480160740 2023-10-16T08:17:50.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.305+0000 7f3486b1d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3480160f20 con 0x7f348015c9f0 2023-10-16T08:17:50.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.305+0000 7f3486b1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34800ccb50 con 0x7f348015cdd0 2023-10-16T08:17:50.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f34850ba640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3480160740 0x7f34800c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59496/0 (socket says 172.21.15.32:59496) 2023-10-16T08:17:50.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f34850ba640 1 -- 172.21.15.32:0/440063556 learned_addr learned my addr 172.21.15.32:0/440063556 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:50.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347ffff640 1 --2- 172.21.15.32:0/440063556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f348015cdd0 0x7f3480069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792500202 0 0) 0x7f3480160c40 con 0x7f3480160740 2023-10-16T08:17:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3470003530 con 0x7f3480160740 2023-10-16T08:17:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763998591 0 0) 0x7f3480160f20 con 0x7f348015c9f0 2023-10-16T08:17:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3480160c40 con 0x7f348015c9f0 2023-10-16T08:17:50.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347ffff640 1 -- 172.21.15.32:0/440063556 >> v1:172.21.15.186:6789/0 conn(0x7f348015c9f0 legacy=0x7f34800cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347ffff640 1 -- 172.21.15.32:0/440063556 >> v1:172.21.15.32:6789/0 conn(0x7f3480160740 legacy=0x7f34800c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347ffff640 1 -- 172.21.15.32:0/440063556 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480161200 con 0x7f348015cdd0 2023-10-16T08:17:50.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.309+0000 7f347ffff640 1 --2- 172.21.15.32:0/440063556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f348015cdd0 0x7f3480069e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f346c0097f0 tx=0x7f346c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdab796492b39c89 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0ac600 con 0x7f348015cdd0 2023-10-16T08:17:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c0a8050 con 0x7f348015cdd0 2023-10-16T08:17:50.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f347f7fe640 1 -- 172.21.15.32:0/440063556 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0ac960 con 0x7f348015cdd0 2023-10-16T08:17:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- 172.21.15.32:0/440063556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f348015cdd0 msgr2=0x7f3480069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 --2- 172.21.15.32:0/440063556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f348015cdd0 0x7f3480069e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f346c0097f0 tx=0x7f346c01c490 comp rx=0 tx=0).stop 2023-10-16T08:17:50.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- 172.21.15.32:0/440063556 shutdown_connections 2023-10-16T08:17:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 --2- 172.21.15.32:0/440063556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f348015cdd0 0x7f3480069e90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- 172.21.15.32:0/440063556 >> 172.21.15.32:0/440063556 conn(0x7f34800b0d10 msgr2=0x7f34800ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- 172.21.15.32:0/440063556 shutdown_connections 2023-10-16T08:17:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- 172.21.15.32:0/440063556 wait complete. 2023-10-16T08:17:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 Processor -- start 2023-10-16T08:17:50.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- start start 2023-10-16T08:17:50.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 0x7f348015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:50.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3480160740 0x7f34800b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:50.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34800c83f0 0x7f34800c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:50.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348014ef20 con 0x7f348015c9f0 2023-10-16T08:17:50.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348014f6c0 con 0x7f34800c83f0 2023-10-16T08:17:50.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f3486b1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348015b7e0 con 0x7f3480160740 2023-10-16T08:17:50.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f34848b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 0x7f348015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:50.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f34848b9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 0x7f348015bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50698/0 (socket says 172.21.15.32:50698) 2023-10-16T08:17:50.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.313+0000 7f34848b9640 1 -- 172.21.15.32:0/991318889 learned_addr learned my addr 172.21.15.32:0/991318889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f347ffff640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3480160740 0x7f34800b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34850ba640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34800c83f0 0x7f34800c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34848b9640 1 -- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3480160740 msgr2=0x7f34800b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34848b9640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3480160740 0x7f34800b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34848b9640 1 -- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34800c83f0 msgr2=0x7f34800c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34848b9640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34800c83f0 0x7f34800c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34848b9640 1 -- 172.21.15.32:0/991318889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c009420 con 0x7f348015c9f0 2023-10-16T08:17:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34850ba640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34800c83f0 0x7f34800c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f34848b9640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 0x7f348015bdd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f347400c490 tx=0x7f347400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34740052d0 con 0x7f348015c9f0 2023-10-16T08:17:50.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474005430 con 0x7f348015c9f0 2023-10-16T08:17:50.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474012870 con 0x7f348015c9f0 2023-10-16T08:17:50.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348014f100 con 0x7f348015c9f0 2023-10-16T08:17:50.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34800b1930 con 0x7f348015c9f0 2023-10-16T08:17:50.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348004e170 con 0x7f348015c9f0 2023-10-16T08:17:50.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.317+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34740129d0 con 0x7f348015c9f0 2023-10-16T08:17:50.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.321+0000 7f347d7fa640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34640664a0 0x7f3464068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:50.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.321+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5552+0+0 (secure 0 0 0) 0x7f34740839a0 con 0x7f348015c9f0 2023-10-16T08:17:50.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.321+0000 7f347ffff640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34640664a0 0x7f3464068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:50.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.321+0000 7f347ffff640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34640664a0 0x7f3464068960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f346c0ae000 tx=0x7f346c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.325+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3474050ed0 con 0x7f348015c9f0 2023-10-16T08:17:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:50.485+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34800c3ad0 con 0x7f348015c9f0 2023-10-16T08:17:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.129+0000 7f347d7fa640 1 -- 172.21.15.32:0/991318889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'testcephfs1_data' created v37) v1 ==== 180+0+0 (secure 0 0 0) 0x7f34740544f0 con 0x7f348015c9f0 2023-10-16T08:17:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_data' created 2023-10-16T08:17:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.133+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34640664a0 msgr2=0x7f3464068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.133+0000 7f3486b1d640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34640664a0 0x7f3464068960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f346c0ae000 tx=0x7f346c008040 comp rx=0 tx=0).stop 2023-10-16T08:17:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.133+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 msgr2=0x7f348015bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:51.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.133+0000 7f3486b1d640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 0x7f348015bdd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f347400c490 tx=0x7f347400c950 comp rx=0 tx=0).stop 2023-10-16T08:17:51.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 shutdown_connections 2023-10-16T08:17:51.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34800c83f0 0x7f34800c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34640664a0 0x7f3464068960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3480160740 0x7f34800b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 --2- 172.21.15.32:0/991318889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348015c9f0 0x7f348015bdd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 >> 172.21.15.32:0/991318889 conn(0x7f34800b0d10 msgr2=0x7f34800c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 shutdown_connections 2023-10-16T08:17:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.137+0000 7f3486b1d640 1 -- 172.21.15.32:0/991318889 wait complete. 2023-10-16T08:17:51.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testcephfs1 testcephfs1_metadata testcephfs1_data 2023-10-16T08:17:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.325+0000 7f75bafc5640 1 Processor -- start 2023-10-16T08:17:51.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.325+0000 7f75bafc5640 1 -- start start 2023-10-16T08:17:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75bafc5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b406d010 0x7f75b40c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75bafc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b40c26b0 0x7f75b40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75bafc5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b415ae30 con 0x7f75b4069e90 2023-10-16T08:17:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75bafc5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b40cb090 con 0x7f75b406d010 2023-10-16T08:17:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75bafc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b406d470 con 0x7f75b40c26b0 2023-10-16T08:17:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b406d010 0x7f75b40c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b406d010 0x7f75b40c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50714/0 (socket says 172.21.15.32:50714) 2023-10-16T08:17:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 -- 172.21.15.32:0/2092891822 learned_addr learned my addr 172.21.15.32:0/2092891822 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b9562640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b40c26b0 0x7f75b40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 -- 172.21.15.32:0/2092891822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b40c26b0 msgr2=0x7f75b40c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b40c26b0 0x7f75b40c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 -- 172.21.15.32:0/2092891822 >> v1:172.21.15.32:6789/0 conn(0x7f75b4069e90 legacy=0x7f75b406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 -- 172.21.15.32:0/2092891822 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40ca510 con 0x7f75b406d010 2023-10-16T08:17:51.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b37fe640 1 -- 172.21.15.32:0/2092891822 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335875614 0 0) 0x7f75b415ae30 con 0x7f75b4069e90 2023-10-16T08:17:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b9562640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b40c26b0 0x7f75b40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.329+0000 7f75b3fff640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b406d010 0x7f75b40c9ce0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a19d8bdc5a00aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b37fe640 1 -- 172.21.15.32:0/2092891822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00a8a30 con 0x7f75b406d010 2023-10-16T08:17:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b37fe640 1 -- 172.21.15.32:0/2092891822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a00a8b90 con 0x7f75b406d010 2023-10-16T08:17:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b37fe640 1 -- 172.21.15.32:0/2092891822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00b0440 con 0x7f75b406d010 2023-10-16T08:17:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- 172.21.15.32:0/2092891822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b406d010 msgr2=0x7f75b40c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b406d010 0x7f75b40c9ce0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a001ed30 comp rx=0 tx=0).stop 2023-10-16T08:17:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- 172.21.15.32:0/2092891822 shutdown_connections 2023-10-16T08:17:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b40c26b0 0x7f75b40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2092891822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b406d010 0x7f75b40c9ce0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- 172.21.15.32:0/2092891822 >> 172.21.15.32:0/2092891822 conn(0x7f75b40b0da0 msgr2=0x7f75b40ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- 172.21.15.32:0/2092891822 shutdown_connections 2023-10-16T08:17:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- 172.21.15.32:0/2092891822 wait complete. 2023-10-16T08:17:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 Processor -- start 2023-10-16T08:17:51.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- start start 2023-10-16T08:17:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b4069e90 0x7f75b41578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75b406d010 0x7f75b40c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 0x7f75b40c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c5e00 con 0x7f75b40c26b0 2023-10-16T08:17:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c8f40 con 0x7f75b406d010 2023-10-16T08:17:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75bafc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4157f30 con 0x7f75b4069e90 2023-10-16T08:17:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b9562640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 0x7f75b40c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b9562640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 0x7f75b40c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50716/0 (socket says 172.21.15.32:50716) 2023-10-16T08:17:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b9562640 1 -- 172.21.15.32:0/2084024408 learned_addr learned my addr 172.21.15.32:0/2084024408 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:51.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b3fff640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75b406d010 0x7f75b40c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.333+0000 7f75b8d61640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b4069e90 0x7f75b41578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b9562640 1 -- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b4069e90 msgr2=0x7f75b41578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b9562640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b4069e90 0x7f75b41578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b9562640 1 -- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75b406d010 msgr2=0x7f75b40c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:51.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b9562640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75b406d010 0x7f75b40c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b9562640 1 -- 172.21.15.32:0/2084024408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00094d0 con 0x7f75b40c26b0 2023-10-16T08:17:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b9562640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 0x7f75b40c8a00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f75ac007f20 tx=0x7f75ac00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00dee0 con 0x7f75b40c26b0 2023-10-16T08:17:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac00f040 con 0x7f75b40c26b0 2023-10-16T08:17:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b4158210 con 0x7f75b40c26b0 2023-10-16T08:17:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac003de0 con 0x7f75b40c26b0 2023-10-16T08:17:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b41587d0 con 0x7f75b40c26b0 2023-10-16T08:17:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b404e170 con 0x7f75b40c26b0 2023-10-16T08:17:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f75ac004030 con 0x7f75b40c26b0 2023-10-16T08:17:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b17fa640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75980664a0 0x7f7598068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.337+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f75ac0843a0 con 0x7f75b40c26b0 2023-10-16T08:17:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.341+0000 7f75b8d61640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75980664a0 0x7f7598068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.341+0000 7f75b8d61640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75980664a0 0x7f7598068960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f75a8000f70 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:51.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.345+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f75ac051690 con 0x7f75b40c26b0 2023-10-16T08:17:51.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:51.533+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"} v 0) v1 -- 0x7f75b40c2090 con 0x7f75b40c26b0 2023-10-16T08:17:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.149+0000 7f75b17fa640 1 -- 172.21.15.32:0/2084024408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 187+0+0 (secure 0 0 0) 0x7f75ac054cb0 con 0x7f75b40c26b0 2023-10-16T08:17:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 5 and data pool 6 2023-10-16T08:17:52.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75980664a0 msgr2=0x7f7598068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75980664a0 0x7f7598068960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f75a8000f70 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2023-10-16T08:17:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 msgr2=0x7f75b40c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 0x7f75b40c8a00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f75ac007f20 tx=0x7f75ac00d560 comp rx=0 tx=0).stop 2023-10-16T08:17:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 shutdown_connections 2023-10-16T08:17:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40c26b0 0x7f75b40c8a00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75b406d010 0x7f75b40c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f75980664a0 0x7f7598068960 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 --2- 172.21.15.32:0/2084024408 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f75b4069e90 0x7f75b41578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 >> 172.21.15.32:0/2084024408 conn(0x7f75b40b0da0 msgr2=0x7f75b40aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 shutdown_connections 2023-10-16T08:17:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.153+0000 7f75bafc5640 1 -- 172.21.15.32:0/2084024408 wait complete. 2023-10-16T08:17:52.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:52.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.325+0000 7fbe0f232640 1 Processor -- start 2023-10-16T08:17:52.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.325+0000 7fbe0f232640 1 -- start start 2023-10-16T08:17:52.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0f232640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbe080c8cb0 0x7fbe080cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0f232640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe080723d0 con 0x7fbe081597c0 2023-10-16T08:17:52.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0f232640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe080717c0 con 0x7fbe08159ba0 2023-10-16T08:17:52.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0f232640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe08071aa0 con 0x7fbe080c8cb0 2023-10-16T08:17:52.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbe081597c0 0x7fbe080ad6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59504/0 (socket says 172.21.15.32:59504) 2023-10-16T08:17:52.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 -- 172.21.15.32:0/2566329449 learned_addr learned my addr 172.21.15.32:0/2566329449 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:52.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0d7cf640 1 --2- 172.21.15.32:0/2566329449 >> v2:172.21.15.186:3300/0 conn(0x7fbe080c8cb0 0x7fbe080cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050065354 0 0) 0x7fbe080723d0 con 0x7fbe081597c0 2023-10-16T08:17:52.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf8005740 con 0x7fbe081597c0 2023-10-16T08:17:52.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426414760 0 0) 0x7fbdf8005740 con 0x7fbe081597c0 2023-10-16T08:17:52.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 >> v2:172.21.15.186:3300/0 conn(0x7fbe080c8cb0 msgr2=0x7fbe080cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:52.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 --2- 172.21.15.32:0/2566329449 >> v2:172.21.15.186:3300/0 conn(0x7fbe080c8cb0 0x7fbe080cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 >> v1:172.21.15.186:6789/0 conn(0x7fbe08159ba0 legacy=0x7fbe080cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08071d80 con 0x7fbe081597c0 2023-10-16T08:17:52.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fbdfc001680 con 0x7fbe081597c0 2023-10-16T08:17:52.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 >> v1:172.21.15.32:6789/0 conn(0x7fbe081597c0 legacy=0x7fbe080ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf8005b90 0x7fbdf8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf80069e0 0x7fbdf8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80073a0 0x7fbdf800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08071aa0 con 0x7fbdf8005b90 2023-10-16T08:17:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf800e240 con 0x7fbdf80073a0 2023-10-16T08:17:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf800e400 con 0x7fbdf80069e0 2023-10-16T08:17:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0d7cf640 1 --2- 172.21.15.32:0/2566329449 >> v2:172.21.15.186:3300/0 conn(0x7fbe080c8cb0 0x7fbe080cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf8005b90 0x7fbdf8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe07fff640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf80069e0 0x7fbdf8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0d7cf640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80073a0 0x7fbdf800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 -- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf80069e0 msgr2=0x7fbdf8006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:52.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf80069e0 0x7fbdf8006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 -- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80073a0 msgr2=0x7fbdf800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:52.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80073a0 0x7fbdf800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0cfce640 1 -- 172.21.15.32:0/2566329449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8005740 con 0x7fbdf8005b90 2023-10-16T08:17:52.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe0d7cf640 1 -- 172.21.15.32:0/2566329449 reap_dead start 2023-10-16T08:17:52.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.329+0000 7fbe07fff640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf80069e0 0x7fbdf8006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:52.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0cfce640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf8005b90 0x7fbdf8005fb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c240 tx=0x7fbdfc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b45fa7f916afb2a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc005ae0 con 0x7fbdf8005b90 2023-10-16T08:17:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc005c40 con 0x7fbdf8005b90 2023-10-16T08:17:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe077fe640 1 -- 172.21.15.32:0/2566329449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0072b0 con 0x7fbdf8005b90 2023-10-16T08:17:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 -- 172.21.15.32:0/2566329449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf8005b90 msgr2=0x7fbdf8005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf8005b90 0x7fbdf8005fb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c240 tx=0x7fbdfc01c9f0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 -- 172.21.15.32:0/2566329449 shutdown_connections 2023-10-16T08:17:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 --2- 172.21.15.32:0/2566329449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf8005b90 0x7fbdf8005fb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 -- 172.21.15.32:0/2566329449 >> 172.21.15.32:0/2566329449 conn(0x7fbe0806c190 msgr2=0x7fbe0806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 -- 172.21.15.32:0/2566329449 shutdown_connections 2023-10-16T08:17:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.333+0000 7fbe0f232640 1 -- 172.21.15.32:0/2566329449 wait complete. 2023-10-16T08:17:52.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 Processor -- start 2023-10-16T08:17:52.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- start start 2023-10-16T08:17:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf8005b90 0x7fbe080c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe08155f50 0x7fbe081574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 0x7fbe081581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08157020 con 0x7fbe081f1bc0 2023-10-16T08:17:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08159010 con 0x7fbe08155f50 2023-10-16T08:17:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08156410 con 0x7fbdf8005b90 2023-10-16T08:17:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 0x7fbe081581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 0x7fbe081581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50736/0 (socket says 172.21.15.32:50736) 2023-10-16T08:17:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 -- 172.21.15.32:0/804196335 learned_addr learned my addr 172.21.15.32:0/804196335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0cfce640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf8005b90 0x7fbe080c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe07fff640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe08155f50 0x7fbe081574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 -- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf8005b90 msgr2=0x7fbe080c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf8005b90 0x7fbe080c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 -- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe08155f50 msgr2=0x7fbe081574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe08155f50 0x7fbe081574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 -- 172.21.15.32:0/804196335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe081566f0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0d7cf640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 0x7fbe081581b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbe080cdff0 tx=0x7fbe00009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000e070 con 0x7fbe081f1bc0 2023-10-16T08:17:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00011da0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000039a0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe081569d0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe081f2050 con 0x7fbe081f1bc0 2023-10-16T08:17:52.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.337+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe081571d0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.341+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe00003b00 con 0x7fbe081f1bc0 2023-10-16T08:17:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.341+0000 7fbe057fa640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbdec0664d0 0x7fbdec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.341+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fbe00084590 con 0x7fbe081f1bc0 2023-10-16T08:17:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.341+0000 7fbe0cfce640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbdec0664d0 0x7fbdec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.341+0000 7fbe0cfce640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbdec0664d0 0x7fbdec068990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0ab000 tx=0x7fbdfc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.345+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe081571d0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.501+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe08074010 con 0x7fbe081f1bc0 2023-10-16T08:17:52.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.501+0000 7fbe057fa640 1 -- 172.21.15.32:0/804196335 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16807 (secure 0 0 0) 0x7fbe000518c0 con 0x7fbe081f1bc0 2023-10-16T08:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:52.509 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:51.540885+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbdec0664d0 msgr2=0x7fbdec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbdec0664d0 0x7fbdec068990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0ab000 tx=0x7fbdfc0a7070 comp rx=0 tx=0).stop 2023-10-16T08:17:52.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 msgr2=0x7fbe081581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 0x7fbe081581b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbe080cdff0 tx=0x7fbe00009f90 comp rx=0 tx=0).stop 2023-10-16T08:17:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 shutdown_connections 2023-10-16T08:17:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe081f1bc0 0x7fbe081581b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe08155f50 0x7fbe081574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbdec0664d0 0x7fbdec068990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 --2- 172.21.15.32:0/804196335 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdf8005b90 0x7fbe080c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 >> 172.21.15.32:0/804196335 conn(0x7fbe0806c190 msgr2=0x7fbe080af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 shutdown_connections 2023-10-16T08:17:52.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.505+0000 7fbe0f232640 1 -- 172.21.15.32:0/804196335 wait complete. 2023-10-16T08:17:52.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe669394640 1 Processor -- start 2023-10-16T08:17:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe669394640 1 -- start start 2023-10-16T08:17:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe669394640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640b3d30 0x7fe6640ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe669394640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6640d0570 con 0x7fe6640b2d50 2023-10-16T08:17:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe669394640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6640cd450 con 0x7fe6640cf8b0 2023-10-16T08:17:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe669394640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640b3130 con 0x7fe6640b3d30 2023-10-16T08:17:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe662ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6640b2d50 0x7fe6640cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59520/0 (socket says 172.21.15.32:59520) 2023-10-16T08:17:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe662ffd640 1 -- 172.21.15.32:0/2708140345 learned_addr learned my addr 172.21.15.32:0/2708140345 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe6637fe640 1 --2- 172.21.15.32:0/2708140345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640b3d30 0x7fe6640ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074011788 0 0) 0x7fe6640d0570 con 0x7fe6640b2d50 2023-10-16T08:17:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe64c003530 con 0x7fe6640b2d50 2023-10-16T08:17:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139353 0 0) 0x7fe6640cd450 con 0x7fe6640cf8b0 2023-10-16T08:17:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6640d0570 con 0x7fe6640cf8b0 2023-10-16T08:17:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe6637fe640 1 -- 172.21.15.32:0/2708140345 >> v1:172.21.15.186:6789/0 conn(0x7fe6640cf8b0 legacy=0x7fe6640cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe6637fe640 1 -- 172.21.15.32:0/2708140345 >> v1:172.21.15.32:6789/0 conn(0x7fe6640b2d50 legacy=0x7fe6640cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.669+0000 7fe6637fe640 1 -- 172.21.15.32:0/2708140345 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6640d07c0 con 0x7fe6640b3d30 2023-10-16T08:17:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe6637fe640 1 --2- 172.21.15.32:0/2708140345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640b3d30 0x7fe6640ce060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009dc0 tx=0x7fe65c01c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f994da6f50d204e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0a5870 con 0x7fe6640b3d30 2023-10-16T08:17:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c0a59d0 con 0x7fe6640b3d30 2023-10-16T08:17:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe661ffb640 1 -- 172.21.15.32:0/2708140345 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0a5d60 con 0x7fe6640b3d30 2023-10-16T08:17:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 -- 172.21.15.32:0/2708140345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640b3d30 msgr2=0x7fe6640ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 --2- 172.21.15.32:0/2708140345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640b3d30 0x7fe6640ce060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009dc0 tx=0x7fe65c01c8a0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 -- 172.21.15.32:0/2708140345 shutdown_connections 2023-10-16T08:17:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 --2- 172.21.15.32:0/2708140345 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640b3d30 0x7fe6640ce060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 -- 172.21.15.32:0/2708140345 >> 172.21.15.32:0/2708140345 conn(0x7fe6640d2790 msgr2=0x7fe6640d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 -- 172.21.15.32:0/2708140345 shutdown_connections 2023-10-16T08:17:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.673+0000 7fe669394640 1 -- 172.21.15.32:0/2708140345 wait complete. 2023-10-16T08:17:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 Processor -- start 2023-10-16T08:17:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- start start 2023-10-16T08:17:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640b2d50 0x7fe66415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 0x7fe66406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640cf8b0 0x7fe66406ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640d1980 con 0x7fe6640b3d30 2023-10-16T08:17:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640c9420 con 0x7fe6640b2d50 2023-10-16T08:17:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66415d4f0 con 0x7fe6640cf8b0 2023-10-16T08:17:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 0x7fe66406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 0x7fe66406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50744/0 (socket says 172.21.15.32:50744) 2023-10-16T08:17:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 -- 172.21.15.32:0/3578303642 learned_addr learned my addr 172.21.15.32:0/3578303642 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe662ffd640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640b2d50 0x7fe66415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6637fe640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640cf8b0 0x7fe66406ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 -- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640cf8b0 msgr2=0x7fe66406ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640cf8b0 0x7fe66406ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 -- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640b2d50 msgr2=0x7fe66415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640b2d50 0x7fe66415eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 -- 172.21.15.32:0/3578303642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c009a70 con 0x7fe6640b3d30 2023-10-16T08:17:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6637fe640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640cf8b0 0x7fe66406ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe6627fc640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 0x7fe66406b530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe65800c550 tx=0x7fe65800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658005480 con 0x7fe6640b3d30 2023-10-16T08:17:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe658003600 con 0x7fe6640b3d30 2023-10-16T08:17:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66415d7d0 con 0x7fe6640b3d30 2023-10-16T08:17:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658003990 con 0x7fe6640b3d30 2023-10-16T08:17:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66415dd90 con 0x7fe6640b3d30 2023-10-16T08:17:52.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.677+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe664073fa0 con 0x7fe6640b3d30 2023-10-16T08:17:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.681+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe658003af0 con 0x7fe6640b3d30 2023-10-16T08:17:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.681+0000 7fe64b7fe640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe6500664a0 0x7fe650068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.681+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fe658083690 con 0x7fe6640b3d30 2023-10-16T08:17:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.681+0000 7fe662ffd640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe6500664a0 0x7fe650068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.681+0000 7fe662ffd640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe6500664a0 0x7fe650068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe6540011f0 tx=0x7fe654009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.685+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe658050910 con 0x7fe6640b3d30 2023-10-16T08:17:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.845+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6640d0b50 con 0x7fe6640b3d30 2023-10-16T08:17:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.845+0000 7fe64b7fe640 1 -- 172.21.15.32:0/3578303642 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16794 (secure 0 0 0) 0x7fe658017090 con 0x7fe6640b3d30 2023-10-16T08:17:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:52.853 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:52.738098+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:52.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe6500664a0 msgr2=0x7fe650068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe6500664a0 0x7fe650068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe6540011f0 tx=0x7fe654009040 comp rx=0 tx=0).stop 2023-10-16T08:17:52.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 msgr2=0x7fe66406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:52.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 0x7fe66406b530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe65800c550 tx=0x7fe65800ca10 comp rx=0 tx=0).stop 2023-10-16T08:17:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 shutdown_connections 2023-10-16T08:17:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe6500664a0 0x7fe650068960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe6640cf8b0 0x7fe66406ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6640b3d30 0x7fe66406b530 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 --2- 172.21.15.32:0/3578303642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6640b2d50 0x7fe66415eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 >> 172.21.15.32:0/3578303642 conn(0x7fe6640d2790 msgr2=0x7fe6640cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 shutdown_connections 2023-10-16T08:17:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:52.849+0000 7fe669394640 1 -- 172.21.15.32:0/3578303642 wait complete. 2023-10-16T08:17:52.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 standby_count_wanted 0 2023-10-16T08:17:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.013+0000 7f4882c91640 1 Processor -- start 2023-10-16T08:17:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4882c91640 1 -- start start 2023-10-16T08:17:53.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4882c91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c165570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:53.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4882c91640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c15ae30 con 0x7f487c069e90 2023-10-16T08:17:53.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4882c91640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c0ca510 con 0x7f487c06d010 2023-10-16T08:17:53.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4882c91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0cb090 con 0x7f487c0c36c0 2023-10-16T08:17:53.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4880a2d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f487c069e90 0x7f487c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60772/0 (socket says 172.21.15.32:60772) 2023-10-16T08:17:53.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f4880a2d640 1 -- 172.21.15.32:0/3762573570 learned_addr learned my addr 172.21.15.32:0/3762573570 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:53.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f488122e640 1 --2- 172.21.15.32:0/3762573570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:53.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285821586 0 0) 0x7f487c15ae30 con 0x7f487c069e90 2023-10-16T08:17:53.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f486c003530 con 0x7f487c069e90 2023-10-16T08:17:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825945175 0 0) 0x7f487c0ca510 con 0x7f487c06d010 2023-10-16T08:17:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c15ae30 con 0x7f487c06d010 2023-10-16T08:17:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f488122e640 1 -- 172.21.15.32:0/3762573570 >> v1:172.21.15.186:6789/0 conn(0x7f487c06d010 legacy=0x7f487c0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f488122e640 1 -- 172.21.15.32:0/3762573570 >> v1:172.21.15.32:6789/0 conn(0x7f487c069e90 legacy=0x7f487c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.017+0000 7f488122e640 1 -- 172.21.15.32:0/3762573570 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0ca7f0 con 0x7f487c0c36c0 2023-10-16T08:17:53.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f488122e640 1 --2- 172.21.15.32:0/3762573570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c165570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4874009dc0 tx=0x7f487401f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9b5c2fd118b703 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740af600 con 0x7f487c0c36c0 2023-10-16T08:17:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740ab050 con 0x7f487c0c36c0 2023-10-16T08:17:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f487b7fe640 1 -- 172.21.15.32:0/3762573570 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740af980 con 0x7f487c0c36c0 2023-10-16T08:17:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 -- 172.21.15.32:0/3762573570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 msgr2=0x7f487c165570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:53.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 --2- 172.21.15.32:0/3762573570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c165570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4874009dc0 tx=0x7f487401f150 comp rx=0 tx=0).stop 2023-10-16T08:17:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 -- 172.21.15.32:0/3762573570 shutdown_connections 2023-10-16T08:17:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 --2- 172.21.15.32:0/3762573570 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c165570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 -- 172.21.15.32:0/3762573570 >> 172.21.15.32:0/3762573570 conn(0x7f487c0b0da0 msgr2=0x7f487c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 -- 172.21.15.32:0/3762573570 shutdown_connections 2023-10-16T08:17:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 -- 172.21.15.32:0/3762573570 wait complete. 2023-10-16T08:17:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 Processor -- start 2023-10-16T08:17:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.021+0000 7f4882c91640 1 -- start start 2023-10-16T08:17:53.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 0x7f487c155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:53.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f487c06d010 0x7f487c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:53.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c0c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:53.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0cbc00 con 0x7f487c069e90 2023-10-16T08:17:53.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c154230 con 0x7f487c06d010 2023-10-16T08:17:53.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1556f0 con 0x7f487c0c36c0 2023-10-16T08:17:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 0x7f487c155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 0x7f487c155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50732/0 (socket says 172.21.15.32:50732) 2023-10-16T08:17:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 -- 172.21.15.32:0/1235023309 learned_addr learned my addr 172.21.15.32:0/1235023309 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f487bfff640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f487c06d010 0x7f487c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:53.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f488122e640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c0c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 -- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 msgr2=0x7f487c0c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c0c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 -- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f487c06d010 msgr2=0x7f487c153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f487c06d010 0x7f487c153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:53.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 -- 172.21.15.32:0/1235023309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874009a70 con 0x7f487c069e90 2023-10-16T08:17:53.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f487bfff640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f487c06d010 0x7f487c153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4880a2d640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 0x7f487c155f60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f487000c4d0 tx=0x7f487000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:53.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870005cf0 con 0x7f487c069e90 2023-10-16T08:17:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870005e50 con 0x7f487c069e90 2023-10-16T08:17:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c1559d0 con 0x7f487c069e90 2023-10-16T08:17:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700072b0 con 0x7f487c069e90 2023-10-16T08:17:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c155cb0 con 0x7f487c069e90 2023-10-16T08:17:53.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.025+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c15c130 con 0x7f487c069e90 2023-10-16T08:17:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.029+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4870007410 con 0x7f487c069e90 2023-10-16T08:17:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.029+0000 7f48797fa640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48600664a0 0x7f4860068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.029+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f48700835e0 con 0x7f487c069e90 2023-10-16T08:17:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.029+0000 7f487bfff640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48600664a0 0x7f4860068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:53.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.029+0000 7f487bfff640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48600664a0 0x7f4860068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f487c0cdcc0 tx=0x7f4868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:53.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.033+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f487c15c130 con 0x7f487c069e90 2023-10-16T08:17:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:53.225+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f487c15c130 con 0x7f487c069e90 2023-10-16T08:17:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.165+0000 7f48797fa640 1 -- 172.21.15.32:0/1235023309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7f487c15c130 con 0x7f487c069e90 2023-10-16T08:17:54.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48600664a0 msgr2=0x7f4860068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:54.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48600664a0 0x7f4860068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f487c0cdcc0 tx=0x7f4868009040 comp rx=0 tx=0).stop 2023-10-16T08:17:54.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 msgr2=0x7f487c155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:54.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 0x7f487c155f60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f487000c4d0 tx=0x7f487000c990 comp rx=0 tx=0).stop 2023-10-16T08:17:54.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 shutdown_connections 2023-10-16T08:17:54.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48600664a0 0x7f4860068960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f487c0c36c0 0x7f487c0c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f487c06d010 0x7f487c153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 --2- 172.21.15.32:0/1235023309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c069e90 0x7f487c155f60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 >> 172.21.15.32:0/1235023309 conn(0x7f487c0b0da0 msgr2=0x7f487c06c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:54.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 shutdown_connections 2023-10-16T08:17:54.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.169+0000 7f4882c91640 1 -- 172.21.15.32:0/1235023309 wait complete. 2023-10-16T08:17:54.197 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T08:17:54.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 allow_standby_replay false 2023-10-16T08:17:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.341+0000 7f6d86668640 1 Processor -- start 2023-10-16T08:17:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d86668640 1 -- start start 2023-10-16T08:17:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d86668640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015ce10 0x7f6d80161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d86668640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6d800cd3e0 0x7f6d800ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d86668640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d800cdc30 con 0x7f6d8006e2f0 2023-10-16T08:17:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d86668640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d800c9750 con 0x7f6d800cd3e0 2023-10-16T08:17:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d86668640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800c99f0 con 0x7f6d8015ce10 2023-10-16T08:17:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d84c05640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d8006e2f0 0x7f6d80069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60788/0 (socket says 172.21.15.32:60788) 2023-10-16T08:17:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d84c05640 1 -- 172.21.15.32:0/3168387325 learned_addr learned my addr 172.21.15.32:0/3168387325 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/3168387325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015ce10 0x7f6d80161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/3168387325 >> v2:172.21.15.186:3300/0 conn(0x7f6d800cd3e0 0x7f6d800ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7effd640 1 -- 172.21.15.32:0/3168387325 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558314063 0 0) 0x7f6d800cdc30 con 0x7f6d8006e2f0 2023-10-16T08:17:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7effd640 1 -- 172.21.15.32:0/3168387325 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d68003530 con 0x7f6d8006e2f0 2023-10-16T08:17:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7ffff640 1 -- 172.21.15.32:0/3168387325 >> v2:172.21.15.186:3300/0 conn(0x7f6d800cd3e0 msgr2=0x7f6d800ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/3168387325 >> v2:172.21.15.186:3300/0 conn(0x7f6d800cd3e0 0x7f6d800ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7ffff640 1 -- 172.21.15.32:0/3168387325 >> v1:172.21.15.32:6789/0 conn(0x7f6d8006e2f0 legacy=0x7f6d80069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.345+0000 7f6d7ffff640 1 -- 172.21.15.32:0/3168387325 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800c9cd0 con 0x7f6d8015ce10 2023-10-16T08:17:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/3168387325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015ce10 0x7f6d80161390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d700097c0 tx=0x7f6d7001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9498300fd4987897 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d7effd640 1 -- 172.21.15.32:0/3168387325 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700ab680 con 0x7f6d8015ce10 2023-10-16T08:17:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d7effd640 1 -- 172.21.15.32:0/3168387325 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d700ab7e0 con 0x7f6d8015ce10 2023-10-16T08:17:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d7effd640 1 -- 172.21.15.32:0/3168387325 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700abb70 con 0x7f6d8015ce10 2023-10-16T08:17:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 -- 172.21.15.32:0/3168387325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015ce10 msgr2=0x7f6d80161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 --2- 172.21.15.32:0/3168387325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015ce10 0x7f6d80161390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d700097c0 tx=0x7f6d7001c480 comp rx=0 tx=0).stop 2023-10-16T08:17:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 -- 172.21.15.32:0/3168387325 shutdown_connections 2023-10-16T08:17:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 --2- 172.21.15.32:0/3168387325 >> v2:172.21.15.186:3300/0 conn(0x7f6d800cd3e0 0x7f6d800ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 --2- 172.21.15.32:0/3168387325 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015ce10 0x7f6d80161390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 -- 172.21.15.32:0/3168387325 >> 172.21.15.32:0/3168387325 conn(0x7f6d800cef20 msgr2=0x7f6d800cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 -- 172.21.15.32:0/3168387325 shutdown_connections 2023-10-16T08:17:54.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 -- 172.21.15.32:0/3168387325 wait complete. 2023-10-16T08:17:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 Processor -- start 2023-10-16T08:17:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.349+0000 7f6d86668640 1 -- start start 2023-10-16T08:17:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8006e2f0 0x7f6d800c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 0x7f6d80153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015f020 0x7f6d8015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8015f9f0 con 0x7f6d800cd3e0 2023-10-16T08:17:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800c4c00 con 0x7f6d8006e2f0 2023-10-16T08:17:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800c3620 con 0x7f6d8015f020 2023-10-16T08:17:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 0x7f6d80153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 0x7f6d80153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50734/0 (socket says 172.21.15.32:50734) 2023-10-16T08:17:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/4285044503 learned_addr learned my addr 172.21.15.32:0/4285044503 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8006e2f0 0x7f6d800c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:54.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d84c05640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015f020 0x7f6d8015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:54.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015f020 msgr2=0x7f6d8015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:54.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015f020 0x7f6d8015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8006e2f0 msgr2=0x7f6d800c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:54.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8006e2f0 0x7f6d800c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/4285044503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70009420 con 0x7f6d800cd3e0 2023-10-16T08:17:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 0x7f6d80153480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e960 tx=0x7f6d7400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74009690 con 0x7f6d800cd3e0 2023-10-16T08:17:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7400ca00 con 0x7f6d800cd3e0 2023-10-16T08:17:54.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400cd70 con 0x7f6d800cd3e0 2023-10-16T08:17:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8006ec90 con 0x7f6d800cd3e0 2023-10-16T08:17:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d800c3be0 con 0x7f6d800cd3e0 2023-10-16T08:17:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.353+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8004e170 con 0x7f6d800cd3e0 2023-10-16T08:17:54.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.357+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6d74013470 con 0x7f6d800cd3e0 2023-10-16T08:17:54.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.357+0000 7f6d7cff9640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d6c0664a0 0x7f6d6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:54.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.357+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f6d74084290 con 0x7f6d800cd3e0 2023-10-16T08:17:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.357+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d6c0664a0 0x7f6d6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.357+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d6c0664a0 0x7f6d6c068960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6d700af000 tx=0x7f6d700a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.361+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6d740515a0 con 0x7f6d800cd3e0 2023-10-16T08:17:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:54.549+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6d8006ef60 con 0x7f6d800cd3e0 2023-10-16T08:17:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d7cff9640 1 -- 172.21.15.32:0/4285044503 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6d74054bc0 con 0x7f6d800cd3e0 2023-10-16T08:17:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d6c0664a0 msgr2=0x7f6d6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d6c0664a0 0x7f6d6c068960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6d700af000 tx=0x7f6d700a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 msgr2=0x7f6d80153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 0x7f6d80153480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e960 tx=0x7f6d7400ee20 comp rx=0 tx=0).stop 2023-10-16T08:17:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 shutdown_connections 2023-10-16T08:17:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d6c0664a0 0x7f6d6c068960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d8015f020 0x7f6d8015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d800cd3e0 0x7f6d80153480 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 --2- 172.21.15.32:0/4285044503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d8006e2f0 0x7f6d800c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.173+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 >> 172.21.15.32:0/4285044503 conn(0x7f6d800cef20 msgr2=0x7f6d8015def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.177+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 shutdown_connections 2023-10-16T08:17:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.177+0000 7f6d86668640 1 -- 172.21.15.32:0/4285044503 wait complete. 2023-10-16T08:17:55.205 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'testcephfs1' 2023-10-16T08:17:55.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:17:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.345+0000 7f2c06165640 1 Processor -- start 2023-10-16T08:17:55.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.345+0000 7f2c06165640 1 -- start start 2023-10-16T08:17:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2c06165640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2c000cf090 0x7f2c000cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2c06165640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000c6ad0 0x7f2c000c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2c06165640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c000c65f0 con 0x7f2c0015ca70 2023-10-16T08:17:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2c06165640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c000cfa60 con 0x7f2c000cf090 2023-10-16T08:17:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2c06165640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000ce820 con 0x7f2c000c6ad0 2023-10-16T08:17:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000c6ad0 0x7f2c000c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfeffd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2c000cf090 0x7f2c000cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bff7fe640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f2c0015ca70 0x7f2c000d3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:53180/0 (socket says 172.21.15.32:53180) 2023-10-16T08:17:55.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bff7fe640 1 -- 172.21.15.32:0/2431901043 learned_addr learned my addr 172.21.15.32:0/2431901043 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfffff640 1 -- 172.21.15.32:0/2431901043 >> v2:172.21.15.186:3300/0 conn(0x7f2c000cf090 msgr2=0x7f2c000cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfffff640 1 --2- 172.21.15.32:0/2431901043 >> v2:172.21.15.186:3300/0 conn(0x7f2c000cf090 0x7f2c000cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfffff640 1 -- 172.21.15.32:0/2431901043 >> v1:172.21.15.186:6789/0 conn(0x7f2c0015ca70 legacy=0x7f2c000d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfe7fc640 1 -- 172.21.15.32:0/2431901043 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983758071 0 0) 0x7f2c000c65f0 con 0x7f2c0015ca70 2023-10-16T08:17:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.349+0000 7f2bfffff640 1 -- 172.21.15.32:0/2431901043 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000cdf80 con 0x7f2c000c6ad0 2023-10-16T08:17:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2bfffff640 1 --2- 172.21.15.32:0/2431901043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000c6ad0 0x7f2c000c6f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009d90 tx=0x7f2bf801c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb3dab0b2e9401e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2bfe7fc640 1 -- 172.21.15.32:0/2431901043 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80a59e0 con 0x7f2c000c6ad0 2023-10-16T08:17:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2bfe7fc640 1 -- 172.21.15.32:0/2431901043 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf80a5b40 con 0x7f2c000c6ad0 2023-10-16T08:17:55.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2bfe7fc640 1 -- 172.21.15.32:0/2431901043 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80ad440 con 0x7f2c000c6ad0 2023-10-16T08:17:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 -- 172.21.15.32:0/2431901043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000c6ad0 msgr2=0x7f2c000c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 --2- 172.21.15.32:0/2431901043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000c6ad0 0x7f2c000c6f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009d90 tx=0x7f2bf801c920 comp rx=0 tx=0).stop 2023-10-16T08:17:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 -- 172.21.15.32:0/2431901043 shutdown_connections 2023-10-16T08:17:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 --2- 172.21.15.32:0/2431901043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000c6ad0 0x7f2c000c6f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 --2- 172.21.15.32:0/2431901043 >> v2:172.21.15.186:3300/0 conn(0x7f2c000cf090 0x7f2c000cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 -- 172.21.15.32:0/2431901043 >> 172.21.15.32:0/2431901043 conn(0x7f2c0009bc50 msgr2=0x7f2c000d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 -- 172.21.15.32:0/2431901043 shutdown_connections 2023-10-16T08:17:55.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 -- 172.21.15.32:0/2431901043 wait complete. 2023-10-16T08:17:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.353+0000 7f2c06165640 1 Processor -- start 2023-10-16T08:17:55.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- start start 2023-10-16T08:17:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c000c6ad0 0x7f2c001590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000cf090 0x7f2c00151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 0x7f2c001528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00152620 con 0x7f2c0015ca70 2023-10-16T08:17:55.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00151d80 con 0x7f2c000c6ad0 2023-10-16T08:17:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00152060 con 0x7f2c000cf090 2023-10-16T08:17:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 0x7f2c001528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bff7fe640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c000c6ad0 0x7f2c001590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 0x7f2c001528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50738/0 (socket says 172.21.15.32:50738) 2023-10-16T08:17:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 -- 172.21.15.32:0/2171558175 learned_addr learned my addr 172.21.15.32:0/2171558175 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfeffd640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000cf090 0x7f2c00151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 -- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000cf090 msgr2=0x7f2c00151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000cf090 0x7f2c00151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 -- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c000c6ad0 msgr2=0x7f2c001590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c000c6ad0 0x7f2c001590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 -- 172.21.15.32:0/2171558175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf8009a70 con 0x7f2c0015ca70 2023-10-16T08:17:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2bfffff640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 0x7f2c001528a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2bf801afd0 tx=0x7f2bf80196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80add90 con 0x7f2c0015ca70 2023-10-16T08:17:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8009450 con 0x7f2c0015ca70 2023-10-16T08:17:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c000d38a0 con 0x7f2c0015ca70 2023-10-16T08:17:55.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80b9670 con 0x7f2c0015ca70 2023-10-16T08:17:55.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c000b5620 con 0x7f2c0015ca70 2023-10-16T08:17:55.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.357+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00074010 con 0x7f2c0015ca70 2023-10-16T08:17:55.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.361+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2bf80b97d0 con 0x7f2c0015ca70 2023-10-16T08:17:55.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.361+0000 7f2be7fff640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bec0664a0 0x7f2bec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.361+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f2bf811f630 con 0x7f2c0015ca70 2023-10-16T08:17:55.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.361+0000 7f2bff7fe640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bec0664a0 0x7f2bec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.361+0000 7f2bff7fe640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bec0664a0 0x7f2bec068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0000fc0 tx=0x7f2bf0000f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:55.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.365+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2bf80eca90 con 0x7f2c0015ca70 2023-10-16T08:17:55.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.553+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c0015a680 con 0x7f2c0015ca70 2023-10-16T08:17:55.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2be7fff640 1 -- 172.21.15.32:0/2171558175 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f2bf80f00b0 con 0x7f2c0015ca70 2023-10-16T08:17:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:55.564 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:17:54.554739+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:17:55.565 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2023-10-16T08:17:55.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bec0664a0 msgr2=0x7f2bec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bec0664a0 0x7f2bec068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0000fc0 tx=0x7f2bf0000f30 comp rx=0 tx=0).stop 2023-10-16T08:17:55.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 msgr2=0x7f2c001528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 0x7f2c001528a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2bf801afd0 tx=0x7f2bf80196c0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 shutdown_connections 2023-10-16T08:17:55.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0015ca70 0x7f2c001528a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2bec0664a0 0x7f2bec068960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2c000cf090 0x7f2c00151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 --2- 172.21.15.32:0/2171558175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c000c6ad0 0x7f2c001590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 >> 172.21.15.32:0/2171558175 conn(0x7f2c0009bc50 msgr2=0x7f2c000b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:55.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 shutdown_connections 2023-10-16T08:17:55.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.557+0000 7f2c06165640 1 -- 172.21.15.32:0/2171558175 wait complete. 2023-10-16T08:17:55.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:17:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.717+0000 7f67e3617640 1 Processor -- start 2023-10-16T08:17:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e3617640 1 -- start start 2023-10-16T08:17:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e3617640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d15d0 0x7f67dc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e3617640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67dc0d3010 0x7f67dc0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e3617640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67dc0ce770 con 0x7f67dc15cb90 2023-10-16T08:17:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e3617640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc0d0600 con 0x7f67dc0d3010 2023-10-16T08:17:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e3617640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0c6450 con 0x7f67dc0d15d0 2023-10-16T08:17:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67dc0d3010 0x7f67dc0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67dc0d3010 0x7f67dc0c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50752/0 (socket says 172.21.15.32:50752) 2023-10-16T08:17:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 -- 172.21.15.32:0/958151003 learned_addr learned my addr 172.21.15.32:0/958151003 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e0bb2640 1 --2- 172.21.15.32:0/958151003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d15d0 0x7f67dc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 -- 172.21.15.32:0/958151003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d15d0 msgr2=0x7f67dc0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 --2- 172.21.15.32:0/958151003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d15d0 0x7f67dc0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 -- 172.21.15.32:0/958151003 >> v1:172.21.15.186:6789/0 conn(0x7f67dc15cb90 legacy=0x7f67dc0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.721+0000 7f67e1bb4640 1 -- 172.21.15.32:0/958151003 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0d08a0 con 0x7f67dc0d3010 2023-10-16T08:17:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e1bb4640 1 --2- 172.21.15.32:0/958151003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d3010 0x7f67dc0c6c10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009e50 tx=0x7f67d401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb48a68b703ce7d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67d3fff640 1 -- 172.21.15.32:0/958151003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d40a5810 con 0x7f67dc0d3010 2023-10-16T08:17:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67d3fff640 1 -- 172.21.15.32:0/958151003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d40a5970 con 0x7f67dc0d3010 2023-10-16T08:17:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67d3fff640 1 -- 172.21.15.32:0/958151003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d40a5d00 con 0x7f67dc0d3010 2023-10-16T08:17:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 -- 172.21.15.32:0/958151003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d3010 msgr2=0x7f67dc0c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 --2- 172.21.15.32:0/958151003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d3010 0x7f67dc0c6c10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009e50 tx=0x7f67d401c950 comp rx=0 tx=0).stop 2023-10-16T08:17:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 -- 172.21.15.32:0/958151003 shutdown_connections 2023-10-16T08:17:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 --2- 172.21.15.32:0/958151003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d3010 0x7f67dc0c6c10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 --2- 172.21.15.32:0/958151003 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d15d0 0x7f67dc0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 -- 172.21.15.32:0/958151003 >> 172.21.15.32:0/958151003 conn(0x7f67dc09bc50 msgr2=0x7f67dc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 -- 172.21.15.32:0/958151003 shutdown_connections 2023-10-16T08:17:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 -- 172.21.15.32:0/958151003 wait complete. 2023-10-16T08:17:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 Processor -- start 2023-10-16T08:17:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.725+0000 7f67e3617640 1 -- start start 2023-10-16T08:17:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 0x7f67dc14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d3010 0x7f67dc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc15cb90 0x7f67dc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc14ee40 con 0x7f67dc0d15d0 2023-10-16T08:17:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0b6b10 con 0x7f67dc15cb90 2023-10-16T08:17:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0caaa0 con 0x7f67dc0d3010 2023-10-16T08:17:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 0x7f67dc14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 0x7f67dc14e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50762/0 (socket says 172.21.15.32:50762) 2023-10-16T08:17:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 -- 172.21.15.32:0/2356012747 learned_addr learned my addr 172.21.15.32:0/2356012747 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e1bb4640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc15cb90 0x7f67dc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e0bb2640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d3010 0x7f67dc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 -- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d3010 msgr2=0x7f67dc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d3010 0x7f67dc0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 -- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc15cb90 msgr2=0x7f67dc1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc15cb90 0x7f67dc1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 -- 172.21.15.32:0/2356012747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009b30 con 0x7f67dc0d15d0 2023-10-16T08:17:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e13b3640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 0x7f67dc14e900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c400 tx=0x7f67d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80052d0 con 0x7f67dc0d15d0 2023-10-16T08:17:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8005430 con 0x7f67dc0d15d0 2023-10-16T08:17:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8003890 con 0x7f67dc0d15d0 2023-10-16T08:17:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc0b5f00 con 0x7f67dc0d15d0 2023-10-16T08:17:55.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc0b64c0 con 0x7f67dc0d15d0 2023-10-16T08:17:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.729+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc074010 con 0x7f67dc0d15d0 2023-10-16T08:17:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.733+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67d80039f0 con 0x7f67dc0d15d0 2023-10-16T08:17:55.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.733+0000 7f67d1ffb640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67c00664a0 0x7f67c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.733+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f67d8083480 con 0x7f67dc0d15d0 2023-10-16T08:17:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.733+0000 7f67e0bb2640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67c00664a0 0x7f67c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.733+0000 7f67e0bb2640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67c00664a0 0x7f67c0068960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f67dc1f81c0 tx=0x7f67cc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:55.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.737+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67d8050850 con 0x7f67dc0d15d0 2023-10-16T08:17:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.893+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67dc0d1f00 con 0x7f67dc0d15d0 2023-10-16T08:17:55.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.893+0000 7f67d1ffb640 1 -- 172.21.15.32:0/2356012747 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16794 (secure 0 0 0) 0x7f67d8053e70 con 0x7f67dc0d15d0 2023-10-16T08:17:55.901 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:55.902 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:17:52.738098+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:17:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67c00664a0 msgr2=0x7f67c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67c00664a0 0x7f67c0068960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f67dc1f81c0 tx=0x7f67cc009190 comp rx=0 tx=0).stop 2023-10-16T08:17:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 msgr2=0x7f67dc14e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 0x7f67dc14e900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c400 tx=0x7f67d800c8c0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 shutdown_connections 2023-10-16T08:17:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67dc15cb90 0x7f67dc1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f67c00664a0 0x7f67c0068960 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f67dc0d3010 0x7f67dc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 --2- 172.21.15.32:0/2356012747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67dc0d15d0 0x7f67dc14e900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 >> 172.21.15.32:0/2356012747 conn(0x7f67dc09bc50 msgr2=0x7f67dc0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:55.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 shutdown_connections 2023-10-16T08:17:55.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:55.897+0000 7f67e3617640 1 -- 172.21.15.32:0/2356012747 wait complete. 2023-10-16T08:17:55.923 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T08:17:55.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:17:56.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 Processor -- start 2023-10-16T08:17:56.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 -- start start 2023-10-16T08:17:56.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f015cb90 0x7f34f00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34f00d3010 0x7f34f00d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00ce770 con 0x7f34f00d1400 2023-10-16T08:17:56.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00d1890 con 0x7f34f00d3010 2023-10-16T08:17:56.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34f5b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00d0600 con 0x7f34f015cb90 2023-10-16T08:17:56.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34ef7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f015cb90 0x7f34f00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34eeffd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f34f00d3010 0x7f34f00d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34effff640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f34f00d1400 0x7f34f00cd3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:53194/0 (socket says 172.21.15.32:53194) 2023-10-16T08:17:56.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34effff640 1 -- 172.21.15.32:0/2540558165 learned_addr learned my addr 172.21.15.32:0/2540558165 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:56.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34ee7fc640 1 -- 172.21.15.32:0/2540558165 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057712863 0 0) 0x7f34f00ce770 con 0x7f34f00d1400 2023-10-16T08:17:56.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34ee7fc640 1 -- 172.21.15.32:0/2540558165 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d8003530 con 0x7f34f00d1400 2023-10-16T08:17:56.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34ef7fe640 1 -- 172.21.15.32:0/2540558165 >> v2:172.21.15.186:3300/0 conn(0x7f34f00d3010 msgr2=0x7f34f00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:56.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34ef7fe640 1 --2- 172.21.15.32:0/2540558165 >> v2:172.21.15.186:3300/0 conn(0x7f34f00d3010 0x7f34f00d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.061+0000 7f34ef7fe640 1 -- 172.21.15.32:0/2540558165 >> v1:172.21.15.186:6789/0 conn(0x7f34f00d1400 legacy=0x7f34f00cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34ef7fe640 1 -- 172.21.15.32:0/2540558165 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00c6450 con 0x7f34f015cb90 2023-10-16T08:17:56.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34eeffd640 1 --2- 172.21.15.32:0/2540558165 >> v2:172.21.15.186:3300/0 conn(0x7f34f00d3010 0x7f34f00d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:56.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34ef7fe640 1 --2- 172.21.15.32:0/2540558165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f015cb90 0x7f34f00cfb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34e00097a0 tx=0x7f34e001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b8f3f5424e9361 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34ee7fc640 1 -- 172.21.15.32:0/2540558165 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00ac600 con 0x7f34f015cb90 2023-10-16T08:17:56.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34ee7fc640 1 -- 172.21.15.32:0/2540558165 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e00a8050 con 0x7f34f015cb90 2023-10-16T08:17:56.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34ee7fc640 1 -- 172.21.15.32:0/2540558165 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00ac960 con 0x7f34f015cb90 2023-10-16T08:17:56.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34f5b3e640 1 -- 172.21.15.32:0/2540558165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f015cb90 msgr2=0x7f34f00cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.065+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/2540558165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f015cb90 0x7f34f00cfb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34e00097a0 tx=0x7f34e001c3a0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- 172.21.15.32:0/2540558165 shutdown_connections 2023-10-16T08:17:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/2540558165 >> v2:172.21.15.186:3300/0 conn(0x7f34f00d3010 0x7f34f00d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/2540558165 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f015cb90 0x7f34f00cfb10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- 172.21.15.32:0/2540558165 >> 172.21.15.32:0/2540558165 conn(0x7f34f009bc50 msgr2=0x7f34f00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:56.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- 172.21.15.32:0/2540558165 shutdown_connections 2023-10-16T08:17:56.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- 172.21.15.32:0/2540558165 wait complete. 2023-10-16T08:17:56.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 Processor -- start 2023-10-16T08:17:56.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- start start 2023-10-16T08:17:56.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f00d1400 0x7f34f014b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 0x7f34f00b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34f015cb90 0x7f34f014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0152dd0 con 0x7f34f00d3010 2023-10-16T08:17:56.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f014bbe0 con 0x7f34f015cb90 2023-10-16T08:17:56.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34f5b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f015a4f0 con 0x7f34f00d1400 2023-10-16T08:17:56.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 0x7f34f00b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 0x7f34f00b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50772/0 (socket says 172.21.15.32:50772) 2023-10-16T08:17:56.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 -- 172.21.15.32:0/3812223312 learned_addr learned my addr 172.21.15.32:0/3812223312 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:56.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34ef7fe640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f00d1400 0x7f34f014b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34effff640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34f015cb90 0x7f34f014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 -- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f00d1400 msgr2=0x7f34f014b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f00d1400 0x7f34f014b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 -- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34f015cb90 msgr2=0x7f34f014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34f015cb90 0x7f34f014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 -- 172.21.15.32:0/3812223312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4009420 con 0x7f34f00d3010 2023-10-16T08:17:56.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.069+0000 7f34eeffd640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 0x7f34f00b5f30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34e400e980 tx=0x7f34e400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400ca00 con 0x7f34f00d3010 2023-10-16T08:17:56.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e400cb60 con 0x7f34f00d3010 2023-10-16T08:17:56.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e00093b0 con 0x7f34f00d3010 2023-10-16T08:17:56.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4013470 con 0x7f34f00d3010 2023-10-16T08:17:56.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f015a740 con 0x7f34f00d3010 2023-10-16T08:17:56.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f0074010 con 0x7f34f00d3010 2023-10-16T08:17:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34e40135d0 con 0x7f34f00d3010 2023-10-16T08:17:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34d7fff640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34dc0664a0 0x7f34dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f34e4084160 con 0x7f34f00d3010 2023-10-16T08:17:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.073+0000 7f34ef7fe640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34dc0664a0 0x7f34dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.077+0000 7f34ef7fe640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34dc0664a0 0x7f34dc068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ae000 tx=0x7f34e00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.077+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34e4051520 con 0x7f34f00d3010 2023-10-16T08:17:56.128 INFO:tasks.ceph.mds.c.smithi032.stderr:2023-10-16T08:17:56.121+0000 7f90095ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:17:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.237+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34f015cf70 con 0x7f34dc0664a0 2023-10-16T08:17:56.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.245+0000 7f34d7fff640 1 -- 172.21.15.32:0/3812223312 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289238 (secure 0 0 0) 0x7f34e0006dc0 con 0x7f34dc0664a0 2023-10-16T08:17:56.254 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:56.258 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:17:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34dc0664a0 msgr2=0x7f34dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34dc0664a0 0x7f34dc068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ae000 tx=0x7f34e00a7000 comp rx=0 tx=0).stop 2023-10-16T08:17:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 msgr2=0x7f34f00b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 0x7f34f00b5f30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34e400e980 tx=0x7f34e400ee40 comp rx=0 tx=0).stop 2023-10-16T08:17:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 shutdown_connections 2023-10-16T08:17:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34f015cb90 0x7f34f014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34f00d3010 0x7f34f00b5f30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f34dc0664a0 0x7f34dc068960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 --2- 172.21.15.32:0/3812223312 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f34f00d1400 0x7f34f014b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 >> 172.21.15.32:0/3812223312 conn(0x7f34f009bc50 msgr2=0x7f34f01559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 shutdown_connections 2023-10-16T08:17:56.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.249+0000 7f34f5b3e640 1 -- 172.21.15.32:0/3812223312 wait complete. 2023-10-16T08:17:56.274 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-10-16T08:17:56.037559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233372,"kb_used_data":4696,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741348,"statfs":{"total":773094113280,"available":772855140352,"internally_reserved":0,"allocated":4808704,"data_stored":2710328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":234016315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.903558"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748795+0000","last_change":"2023-10-16T08:17:51.761871+0000","last_active":"2023-10-16T08:17:52.748795+0000","last_peered":"2023-10-16T08:17:52.748795+0000","last_clean":"2023-10-16T08:17:52.748795+0000","last_became_active":"2023-10-16T08:17:51.761568+0000","last_became_peered":"2023-10-16T08:17:51.761568+0000","last_unstale":"2023-10-16T08:17:52.748795+0000","last_undegraded":"2023-10-16T08:17:52.748795+0000","last_fullsized":"2023-10-16T08:17:52.748795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:25:30.639378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748694+0000","last_change":"2023-10-16T08:17:51.765816+0000","last_active":"2023-10-16T08:17:52.748694+0000","last_peered":"2023-10-16T08:17:52.748694+0000","last_clean":"2023-10-16T08:17:52.748694+0000","last_became_active":"2023-10-16T08:17:51.765431+0000","last_became_peered":"2023-10-16T08:17:51.765431+0000","last_unstale":"2023-10-16T08:17:52.748694+0000","last_undegraded":"2023-10-16T08:17:52.748694+0000","last_fullsized":"2023-10-16T08:17:52.748694+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:09.245608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525493+0000","last_change":"2023-10-16T08:17:51.763403+0000","last_active":"2023-10-16T08:17:53.525493+0000","last_peered":"2023-10-16T08:17:53.525493+0000","last_clean":"2023-10-16T08:17:53.525493+0000","last_became_active":"2023-10-16T08:17:51.756500+0000","last_became_peered":"2023-10-16T08:17:51.756500+0000","last_unstale":"2023-10-16T08:17:53.525493+0000","last_undegraded":"2023-10-16T08:17:53.525493+0000","last_fullsized":"2023-10-16T08:17:53.525493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:16.234591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605971+0000","last_change":"2023-10-16T08:17:51.771620+0000","last_active":"2023-10-16T08:17:53.605971+0000","last_peered":"2023-10-16T08:17:53.605971+0000","last_clean":"2023-10-16T08:17:53.605971+0000","last_became_active":"2023-10-16T08:17:51.771350+0000","last_became_peered":"2023-10-16T08:17:51.771350+0000","last_unstale":"2023-10-16T08:17:53.605971+0000","last_undegraded":"2023-10-16T08:17:53.605971+0000","last_fullsized":"2023-10-16T08:17:53.605971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:05.549988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605730+0000","last_change":"2023-10-16T08:17:51.770594+0000","last_active":"2023-10-16T08:17:53.605730+0000","last_peered":"2023-10-16T08:17:53.605730+0000","last_clean":"2023-10-16T08:17:53.605730+0000","last_became_active":"2023-10-16T08:17:51.770322+0000","last_became_peered":"2023-10-16T08:17:51.770322+0000","last_unstale":"2023-10-16T08:17:53.605730+0000","last_undegraded":"2023-10-16T08:17:53.605730+0000","last_fullsized":"2023-10-16T08:17:53.605730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:02:28.473257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747574+0000","last_change":"2023-10-16T08:17:51.752780+0000","last_active":"2023-10-16T08:17:52.747574+0000","last_peered":"2023-10-16T08:17:52.747574+0000","last_clean":"2023-10-16T08:17:52.747574+0000","last_became_active":"2023-10-16T08:17:51.752519+0000","last_became_peered":"2023-10-16T08:17:51.752519+0000","last_unstale":"2023-10-16T08:17:52.747574+0000","last_undegraded":"2023-10-16T08:17:52.747574+0000","last_fullsized":"2023-10-16T08:17:52.747574+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:52.230821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608026+0000","last_change":"2023-10-16T08:17:51.760424+0000","last_active":"2023-10-16T08:17:53.608026+0000","last_peered":"2023-10-16T08:17:53.608026+0000","last_clean":"2023-10-16T08:17:53.608026+0000","last_became_active":"2023-10-16T08:17:51.758777+0000","last_became_peered":"2023-10-16T08:17:51.758777+0000","last_unstale":"2023-10-16T08:17:53.608026+0000","last_undegraded":"2023-10-16T08:17:53.608026+0000","last_fullsized":"2023-10-16T08:17:53.608026+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:56.972901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748274+0000","last_change":"2023-10-16T08:17:51.767472+0000","last_active":"2023-10-16T08:17:52.748274+0000","last_peered":"2023-10-16T08:17:52.748274+0000","last_clean":"2023-10-16T08:17:52.748274+0000","last_became_active":"2023-10-16T08:17:51.767135+0000","last_became_peered":"2023-10-16T08:17:51.767135+0000","last_unstale":"2023-10-16T08:17:52.748274+0000","last_undegraded":"2023-10-16T08:17:52.748274+0000","last_fullsized":"2023-10-16T08:17:52.748274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:30.638593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525747+0000","last_change":"2023-10-16T08:17:51.756513+0000","last_active":"2023-10-16T08:17:53.525747+0000","last_peered":"2023-10-16T08:17:53.525747+0000","last_clean":"2023-10-16T08:17:53.525747+0000","last_became_active":"2023-10-16T08:17:51.756269+0000","last_became_peered":"2023-10-16T08:17:51.756269+0000","last_unstale":"2023-10-16T08:17:53.525747+0000","last_undegraded":"2023-10-16T08:17:53.525747+0000","last_fullsized":"2023-10-16T08:17:53.525747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:49.511834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608376+0000","last_change":"2023-10-16T08:17:51.765803+0000","last_active":"2023-10-16T08:17:53.608376+0000","last_peered":"2023-10-16T08:17:53.608376+0000","last_clean":"2023-10-16T08:17:53.608376+0000","last_became_active":"2023-10-16T08:17:51.765526+0000","last_became_peered":"2023-10-16T08:17:51.765526+0000","last_unstale":"2023-10-16T08:17:53.608376+0000","last_undegraded":"2023-10-16T08:17:53.608376+0000","last_fullsized":"2023-10-16T08:17:53.608376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:18.074756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608631+0000","last_change":"2023-10-16T08:17:51.767369+0000","last_active":"2023-10-16T08:17:53.608631+0000","last_peered":"2023-10-16T08:17:53.608631+0000","last_clean":"2023-10-16T08:17:53.608631+0000","last_became_active":"2023-10-16T08:17:51.767099+0000","last_became_peered":"2023-10-16T08:17:51.767099+0000","last_unstale":"2023-10-16T08:17:53.608631+0000","last_undegraded":"2023-10-16T08:17:53.608631+0000","last_fullsized":"2023-10-16T08:17:53.608631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:30.638593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525459+0000","last_change":"2023-10-16T08:17:51.757496+0000","last_active":"2023-10-16T08:17:53.525459+0000","last_peered":"2023-10-16T08:17:53.525459+0000","last_clean":"2023-10-16T08:17:53.525459+0000","last_became_active":"2023-10-16T08:17:51.756308+0000","last_became_peered":"2023-10-16T08:17:51.756308+0000","last_unstale":"2023-10-16T08:17:53.525459+0000","last_undegraded":"2023-10-16T08:17:53.525459+0000","last_fullsized":"2023-10-16T08:17:53.525459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:11.855807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609267+0000","last_change":"2023-10-16T08:17:51.766543+0000","last_active":"2023-10-16T08:17:53.609267+0000","last_peered":"2023-10-16T08:17:53.609267+0000","last_clean":"2023-10-16T08:17:53.609267+0000","last_became_active":"2023-10-16T08:17:51.766177+0000","last_became_peered":"2023-10-16T08:17:51.766177+0000","last_unstale":"2023-10-16T08:17:53.609267+0000","last_undegraded":"2023-10-16T08:17:53.609267+0000","last_fullsized":"2023-10-16T08:17:53.609267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:44.565473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607655+0000","last_change":"2023-10-16T08:17:51.771095+0000","last_active":"2023-10-16T08:17:53.607655+0000","last_peered":"2023-10-16T08:17:53.607655+0000","last_clean":"2023-10-16T08:17:53.607655+0000","last_became_active":"2023-10-16T08:17:51.770786+0000","last_became_peered":"2023-10-16T08:17:51.770786+0000","last_unstale":"2023-10-16T08:17:53.607655+0000","last_undegraded":"2023-10-16T08:17:53.607655+0000","last_fullsized":"2023-10-16T08:17:53.607655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:35.540947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608715+0000","last_change":"2023-10-16T08:17:51.766213+0000","last_active":"2023-10-16T08:17:53.608715+0000","last_peered":"2023-10-16T08:17:53.608715+0000","last_clean":"2023-10-16T08:17:53.608715+0000","last_became_active":"2023-10-16T08:17:51.765857+0000","last_became_peered":"2023-10-16T08:17:51.765857+0000","last_unstale":"2023-10-16T08:17:53.608715+0000","last_undegraded":"2023-10-16T08:17:53.608715+0000","last_fullsized":"2023-10-16T08:17:53.608715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:08.178740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607307+0000","last_change":"2023-10-16T08:17:51.760331+0000","last_active":"2023-10-16T08:17:53.607307+0000","last_peered":"2023-10-16T08:17:53.607307+0000","last_clean":"2023-10-16T08:17:53.607307+0000","last_became_active":"2023-10-16T08:17:51.760013+0000","last_became_peered":"2023-10-16T08:17:51.760013+0000","last_unstale":"2023-10-16T08:17:53.607307+0000","last_undegraded":"2023-10-16T08:17:53.607307+0000","last_fullsized":"2023-10-16T08:17:53.607307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:10.704012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608456+0000","last_change":"2023-10-16T08:17:51.757160+0000","last_active":"2023-10-16T08:17:53.608456+0000","last_peered":"2023-10-16T08:17:53.608456+0000","last_clean":"2023-10-16T08:17:53.608456+0000","last_became_active":"2023-10-16T08:17:51.756344+0000","last_became_peered":"2023-10-16T08:17:51.756344+0000","last_unstale":"2023-10-16T08:17:53.608456+0000","last_undegraded":"2023-10-16T08:17:53.608456+0000","last_fullsized":"2023-10-16T08:17:53.608456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:18.074756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749338+0000","last_change":"2023-10-16T08:17:51.773910+0000","last_active":"2023-10-16T08:17:52.749338+0000","last_peered":"2023-10-16T08:17:52.749338+0000","last_clean":"2023-10-16T08:17:52.749338+0000","last_became_active":"2023-10-16T08:17:51.773592+0000","last_became_peered":"2023-10-16T08:17:51.773592+0000","last_unstale":"2023-10-16T08:17:52.749338+0000","last_undegraded":"2023-10-16T08:17:52.749338+0000","last_fullsized":"2023-10-16T08:17:52.749338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:25.146020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.606109+0000","last_change":"2023-10-16T08:17:51.762446+0000","last_active":"2023-10-16T08:17:53.606109+0000","last_peered":"2023-10-16T08:17:53.606109+0000","last_clean":"2023-10-16T08:17:53.606109+0000","last_became_active":"2023-10-16T08:17:51.762174+0000","last_became_peered":"2023-10-16T08:17:51.762174+0000","last_unstale":"2023-10-16T08:17:53.606109+0000","last_undegraded":"2023-10-16T08:17:53.606109+0000","last_fullsized":"2023-10-16T08:17:53.606109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:49.836706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749295+0000","last_change":"2023-10-16T08:17:51.772967+0000","last_active":"2023-10-16T08:17:52.749295+0000","last_peered":"2023-10-16T08:17:52.749295+0000","last_clean":"2023-10-16T08:17:52.749295+0000","last_became_active":"2023-10-16T08:17:51.772195+0000","last_became_peered":"2023-10-16T08:17:51.772195+0000","last_unstale":"2023-10-16T08:17:52.749295+0000","last_undegraded":"2023-10-16T08:17:52.749295+0000","last_fullsized":"2023-10-16T08:17:52.749295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:49.836706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609392+0000","last_change":"2023-10-16T08:17:51.754447+0000","last_active":"2023-10-16T08:17:53.609392+0000","last_peered":"2023-10-16T08:17:53.609392+0000","last_clean":"2023-10-16T08:17:53.609392+0000","last_became_active":"2023-10-16T08:17:51.754098+0000","last_became_peered":"2023-10-16T08:17:51.754098+0000","last_unstale":"2023-10-16T08:17:53.609392+0000","last_undegraded":"2023-10-16T08:17:53.609392+0000","last_fullsized":"2023-10-16T08:17:53.609392+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:09.245608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608032+0000","last_change":"2023-10-16T08:17:51.771524+0000","last_active":"2023-10-16T08:17:53.608032+0000","last_peered":"2023-10-16T08:17:53.608032+0000","last_clean":"2023-10-16T08:17:53.608032+0000","last_became_active":"2023-10-16T08:17:51.770806+0000","last_became_peered":"2023-10-16T08:17:51.770806+0000","last_unstale":"2023-10-16T08:17:53.608032+0000","last_undegraded":"2023-10-16T08:17:53.608032+0000","last_fullsized":"2023-10-16T08:17:53.608032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:11.855807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607437+0000","last_change":"2023-10-16T08:17:51.765141+0000","last_active":"2023-10-16T08:17:53.607437+0000","last_peered":"2023-10-16T08:17:53.607437+0000","last_clean":"2023-10-16T08:17:53.607437+0000","last_became_active":"2023-10-16T08:17:51.764755+0000","last_became_peered":"2023-10-16T08:17:51.764755+0000","last_unstale":"2023-10-16T08:17:53.607437+0000","last_undegraded":"2023-10-16T08:17:53.607437+0000","last_fullsized":"2023-10-16T08:17:53.607437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:49.511834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605752+0000","last_change":"2023-10-16T08:17:51.768598+0000","last_active":"2023-10-16T08:17:53.605752+0000","last_peered":"2023-10-16T08:17:53.605752+0000","last_clean":"2023-10-16T08:17:53.605752+0000","last_became_active":"2023-10-16T08:17:51.768382+0000","last_became_peered":"2023-10-16T08:17:51.768382+0000","last_unstale":"2023-10-16T08:17:53.605752+0000","last_undegraded":"2023-10-16T08:17:53.605752+0000","last_fullsized":"2023-10-16T08:17:53.605752+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:46.209697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749505+0000","last_change":"2023-10-16T08:17:51.760433+0000","last_active":"2023-10-16T08:17:52.749505+0000","last_peered":"2023-10-16T08:17:52.749505+0000","last_clean":"2023-10-16T08:17:52.749505+0000","last_became_active":"2023-10-16T08:17:51.760140+0000","last_became_peered":"2023-10-16T08:17:51.760140+0000","last_unstale":"2023-10-16T08:17:52.749505+0000","last_undegraded":"2023-10-16T08:17:52.749505+0000","last_fullsized":"2023-10-16T08:17:52.749505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:35.540947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747917+0000","last_change":"2023-10-16T08:17:51.760272+0000","last_active":"2023-10-16T08:17:52.747917+0000","last_peered":"2023-10-16T08:17:52.747917+0000","last_clean":"2023-10-16T08:17:52.747917+0000","last_became_active":"2023-10-16T08:17:51.758030+0000","last_became_peered":"2023-10-16T08:17:51.758030+0000","last_unstale":"2023-10-16T08:17:52.747917+0000","last_undegraded":"2023-10-16T08:17:52.747917+0000","last_fullsized":"2023-10-16T08:17:52.747917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:49.511834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749372+0000","last_change":"2023-10-16T08:17:51.764293+0000","last_active":"2023-10-16T08:17:52.749372+0000","last_peered":"2023-10-16T08:17:52.749372+0000","last_clean":"2023-10-16T08:17:52.749372+0000","last_became_active":"2023-10-16T08:17:51.763961+0000","last_became_peered":"2023-10-16T08:17:51.763961+0000","last_unstale":"2023-10-16T08:17:52.749372+0000","last_undegraded":"2023-10-16T08:17:52.749372+0000","last_fullsized":"2023-10-16T08:17:52.749372+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:42:12.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608260+0000","last_change":"2023-10-16T08:17:51.763333+0000","last_active":"2023-10-16T08:17:53.608260+0000","last_peered":"2023-10-16T08:17:53.608260+0000","last_clean":"2023-10-16T08:17:53.608260+0000","last_became_active":"2023-10-16T08:17:51.762447+0000","last_became_peered":"2023-10-16T08:17:51.762447+0000","last_unstale":"2023-10-16T08:17:53.608260+0000","last_undegraded":"2023-10-16T08:17:53.608260+0000","last_fullsized":"2023-10-16T08:17:53.608260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:33.405605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749066+0000","last_change":"2023-10-16T08:17:51.752557+0000","last_active":"2023-10-16T08:17:52.749066+0000","last_peered":"2023-10-16T08:17:52.749066+0000","last_clean":"2023-10-16T08:17:52.749066+0000","last_became_active":"2023-10-16T08:17:51.752286+0000","last_became_peered":"2023-10-16T08:17:51.752286+0000","last_unstale":"2023-10-16T08:17:52.749066+0000","last_undegraded":"2023-10-16T08:17:52.749066+0000","last_fullsized":"2023-10-16T08:17:52.749066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:05.549988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749172+0000","last_change":"2023-10-16T08:17:51.753662+0000","last_active":"2023-10-16T08:17:52.749172+0000","last_peered":"2023-10-16T08:17:52.749172+0000","last_clean":"2023-10-16T08:17:52.749172+0000","last_became_active":"2023-10-16T08:17:51.753380+0000","last_became_peered":"2023-10-16T08:17:51.753380+0000","last_unstale":"2023-10-16T08:17:52.749172+0000","last_undegraded":"2023-10-16T08:17:52.749172+0000","last_fullsized":"2023-10-16T08:17:52.749172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:38.864578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607858+0000","last_change":"2023-10-16T08:17:51.769094+0000","last_active":"2023-10-16T08:17:53.607858+0000","last_peered":"2023-10-16T08:17:53.607858+0000","last_clean":"2023-10-16T08:17:53.607858+0000","last_became_active":"2023-10-16T08:17:51.768850+0000","last_became_peered":"2023-10-16T08:17:51.768850+0000","last_unstale":"2023-10-16T08:17:53.607858+0000","last_undegraded":"2023-10-16T08:17:53.607858+0000","last_fullsized":"2023-10-16T08:17:53.607858+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:56.972901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747229+0000","last_change":"2023-10-16T08:17:51.764687+0000","last_active":"2023-10-16T08:17:52.747229+0000","last_peered":"2023-10-16T08:17:52.747229+0000","last_clean":"2023-10-16T08:17:52.747229+0000","last_became_active":"2023-10-16T08:17:51.764386+0000","last_became_peered":"2023-10-16T08:17:51.764386+0000","last_unstale":"2023-10-16T08:17:52.747229+0000","last_undegraded":"2023-10-16T08:17:52.747229+0000","last_fullsized":"2023-10-16T08:17:52.747229+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:11.855807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608541+0000","last_change":"2023-10-16T08:17:51.764566+0000","last_active":"2023-10-16T08:17:53.608541+0000","last_peered":"2023-10-16T08:17:53.608541+0000","last_clean":"2023-10-16T08:17:53.608541+0000","last_became_active":"2023-10-16T08:17:51.763026+0000","last_became_peered":"2023-10-16T08:17:51.763026+0000","last_unstale":"2023-10-16T08:17:53.608541+0000","last_undegraded":"2023-10-16T08:17:53.608541+0000","last_fullsized":"2023-10-16T08:17:53.608541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:33.756951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749545+0000","last_change":"2023-10-16T08:17:51.166289+0000","last_active":"2023-10-16T08:17:52.749545+0000","last_peered":"2023-10-16T08:17:52.749545+0000","last_clean":"2023-10-16T08:17:52.749545+0000","last_became_active":"2023-10-16T08:17:51.166015+0000","last_became_peered":"2023-10-16T08:17:51.166015+0000","last_unstale":"2023-10-16T08:17:52.749545+0000","last_undegraded":"2023-10-16T08:17:52.749545+0000","last_fullsized":"2023-10-16T08:17:52.749545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:45.202816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747874+0000","last_change":"2023-10-16T08:17:51.755280+0000","last_active":"2023-10-16T08:17:52.747874+0000","last_peered":"2023-10-16T08:17:52.747874+0000","last_clean":"2023-10-16T08:17:52.747874+0000","last_became_active":"2023-10-16T08:17:51.754983+0000","last_became_peered":"2023-10-16T08:17:51.754983+0000","last_unstale":"2023-10-16T08:17:52.747874+0000","last_undegraded":"2023-10-16T08:17:52.747874+0000","last_fullsized":"2023-10-16T08:17:52.747874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:10.704012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525629+0000","last_change":"2023-10-16T08:17:51.157530+0000","last_active":"2023-10-16T08:17:53.525629+0000","last_peered":"2023-10-16T08:17:53.525629+0000","last_clean":"2023-10-16T08:17:53.525629+0000","last_became_active":"2023-10-16T08:17:51.157236+0000","last_became_peered":"2023-10-16T08:17:51.157236+0000","last_unstale":"2023-10-16T08:17:53.525629+0000","last_undegraded":"2023-10-16T08:17:53.525629+0000","last_fullsized":"2023-10-16T08:17:53.525629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:51.087792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608911+0000","last_change":"2023-10-16T08:17:51.770479+0000","last_active":"2023-10-16T08:17:53.608911+0000","last_peered":"2023-10-16T08:17:53.608911+0000","last_clean":"2023-10-16T08:17:53.608911+0000","last_became_active":"2023-10-16T08:17:51.769622+0000","last_became_peered":"2023-10-16T08:17:51.769622+0000","last_unstale":"2023-10-16T08:17:53.608911+0000","last_undegraded":"2023-10-16T08:17:53.608911+0000","last_fullsized":"2023-10-16T08:17:53.608911+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:50.948998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748037+0000","last_change":"2023-10-16T08:17:51.166015+0000","last_active":"2023-10-16T08:17:52.748037+0000","last_peered":"2023-10-16T08:17:52.748037+0000","last_clean":"2023-10-16T08:17:52.748037+0000","last_became_active":"2023-10-16T08:17:51.165599+0000","last_became_peered":"2023-10-16T08:17:51.165599+0000","last_unstale":"2023-10-16T08:17:52.748037+0000","last_undegraded":"2023-10-16T08:17:52.748037+0000","last_fullsized":"2023-10-16T08:17:52.748037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:05.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748753+0000","last_change":"2023-10-16T08:17:51.760600+0000","last_active":"2023-10-16T08:17:52.748753+0000","last_peered":"2023-10-16T08:17:52.748753+0000","last_clean":"2023-10-16T08:17:52.748753+0000","last_became_active":"2023-10-16T08:17:51.760353+0000","last_became_peered":"2023-10-16T08:17:51.760353+0000","last_unstale":"2023-10-16T08:17:52.748753+0000","last_undegraded":"2023-10-16T08:17:52.748753+0000","last_fullsized":"2023-10-16T08:17:52.748753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:58:34.233023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608467+0000","last_change":"2023-10-16T08:17:51.163541+0000","last_active":"2023-10-16T08:17:53.608467+0000","last_peered":"2023-10-16T08:17:53.608467+0000","last_clean":"2023-10-16T08:17:53.608467+0000","last_became_active":"2023-10-16T08:17:51.163144+0000","last_became_peered":"2023-10-16T08:17:51.163144+0000","last_unstale":"2023-10-16T08:17:53.608467+0000","last_undegraded":"2023-10-16T08:17:53.608467+0000","last_fullsized":"2023-10-16T08:17:53.608467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:48.829825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525511+0000","last_change":"2023-10-16T08:17:51.746836+0000","last_active":"2023-10-16T08:17:53.525511+0000","last_peered":"2023-10-16T08:17:53.525511+0000","last_clean":"2023-10-16T08:17:53.525511+0000","last_became_active":"2023-10-16T08:17:51.746096+0000","last_became_peered":"2023-10-16T08:17:51.746096+0000","last_unstale":"2023-10-16T08:17:53.525511+0000","last_undegraded":"2023-10-16T08:17:53.525511+0000","last_fullsized":"2023-10-16T08:17:53.525511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:52.230821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609396+0000","last_change":"2023-10-16T08:17:51.166500+0000","last_active":"2023-10-16T08:17:53.609396+0000","last_peered":"2023-10-16T08:17:53.609396+0000","last_clean":"2023-10-16T08:17:53.609396+0000","last_became_active":"2023-10-16T08:17:51.166141+0000","last_became_peered":"2023-10-16T08:17:51.166141+0000","last_unstale":"2023-10-16T08:17:53.609396+0000","last_undegraded":"2023-10-16T08:17:53.609396+0000","last_fullsized":"2023-10-16T08:17:53.609396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:51.223940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608843+0000","last_change":"2023-10-16T08:17:51.754413+0000","last_active":"2023-10-16T08:17:53.608843+0000","last_peered":"2023-10-16T08:17:53.608843+0000","last_clean":"2023-10-16T08:17:53.608843+0000","last_became_active":"2023-10-16T08:17:51.754054+0000","last_became_peered":"2023-10-16T08:17:51.754054+0000","last_unstale":"2023-10-16T08:17:53.608843+0000","last_undegraded":"2023-10-16T08:17:53.608843+0000","last_fullsized":"2023-10-16T08:17:53.608843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:57.348350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607988+0000","last_change":"2023-10-16T08:17:51.168880+0000","last_active":"2023-10-16T08:17:53.607988+0000","last_peered":"2023-10-16T08:17:53.607988+0000","last_clean":"2023-10-16T08:17:53.607988+0000","last_became_active":"2023-10-16T08:17:51.168449+0000","last_became_peered":"2023-10-16T08:17:51.168449+0000","last_unstale":"2023-10-16T08:17:53.607988+0000","last_undegraded":"2023-10-16T08:17:53.607988+0000","last_fullsized":"2023-10-16T08:17:53.607988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:34.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608218+0000","last_change":"2023-10-16T08:17:51.772018+0000","last_active":"2023-10-16T08:17:53.608218+0000","last_peered":"2023-10-16T08:17:53.608218+0000","last_clean":"2023-10-16T08:17:53.608218+0000","last_became_active":"2023-10-16T08:17:51.771721+0000","last_became_peered":"2023-10-16T08:17:51.771721+0000","last_unstale":"2023-10-16T08:17:53.608218+0000","last_undegraded":"2023-10-16T08:17:53.608218+0000","last_fullsized":"2023-10-16T08:17:53.608218+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:16.234591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747586+0000","last_change":"2023-10-16T08:17:51.156660+0000","last_active":"2023-10-16T08:17:52.747586+0000","last_peered":"2023-10-16T08:17:52.747586+0000","last_clean":"2023-10-16T08:17:52.747586+0000","last_became_active":"2023-10-16T08:17:51.156227+0000","last_became_peered":"2023-10-16T08:17:51.156227+0000","last_unstale":"2023-10-16T08:17:52.747586+0000","last_undegraded":"2023-10-16T08:17:52.747586+0000","last_fullsized":"2023-10-16T08:17:52.747586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:48.829825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748485+0000","last_change":"2023-10-16T08:17:51.764715+0000","last_active":"2023-10-16T08:17:52.748485+0000","last_peered":"2023-10-16T08:17:52.748485+0000","last_clean":"2023-10-16T08:17:52.748485+0000","last_became_active":"2023-10-16T08:17:51.764432+0000","last_became_peered":"2023-10-16T08:17:51.764432+0000","last_unstale":"2023-10-16T08:17:52.748485+0000","last_undegraded":"2023-10-16T08:17:52.748485+0000","last_fullsized":"2023-10-16T08:17:52.748485+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:38.450749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607388+0000","last_change":"2023-10-16T08:17:51.163200+0000","last_active":"2023-10-16T08:17:53.607388+0000","last_peered":"2023-10-16T08:17:53.607388+0000","last_clean":"2023-10-16T08:17:53.607388+0000","last_became_active":"2023-10-16T08:17:51.162800+0000","last_became_peered":"2023-10-16T08:17:51.162800+0000","last_unstale":"2023-10-16T08:17:53.607388+0000","last_undegraded":"2023-10-16T08:17:53.607388+0000","last_fullsized":"2023-10-16T08:17:53.607388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:37.857697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605975+0000","last_change":"2023-10-16T08:17:51.770675+0000","last_active":"2023-10-16T08:17:53.605975+0000","last_peered":"2023-10-16T08:17:53.605975+0000","last_clean":"2023-10-16T08:17:53.605975+0000","last_became_active":"2023-10-16T08:17:51.770356+0000","last_became_peered":"2023-10-16T08:17:51.770356+0000","last_unstale":"2023-10-16T08:17:53.605975+0000","last_undegraded":"2023-10-16T08:17:53.605975+0000","last_fullsized":"2023-10-16T08:17:53.605975+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:32.266812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748800+0000","last_change":"2023-10-16T08:17:51.158796+0000","last_active":"2023-10-16T08:17:52.748800+0000","last_peered":"2023-10-16T08:17:52.748800+0000","last_clean":"2023-10-16T08:17:52.748800+0000","last_became_active":"2023-10-16T08:17:51.158083+0000","last_became_peered":"2023-10-16T08:17:51.158083+0000","last_unstale":"2023-10-16T08:17:52.748800+0000","last_undegraded":"2023-10-16T08:17:52.748800+0000","last_fullsized":"2023-10-16T08:17:52.748800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:32.750070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608674+0000","last_change":"2023-10-16T08:17:51.759165+0000","last_active":"2023-10-16T08:17:53.608674+0000","last_peered":"2023-10-16T08:17:53.608674+0000","last_clean":"2023-10-16T08:17:53.608674+0000","last_became_active":"2023-10-16T08:17:51.758877+0000","last_became_peered":"2023-10-16T08:17:51.758877+0000","last_unstale":"2023-10-16T08:17:53.608674+0000","last_undegraded":"2023-10-16T08:17:53.608674+0000","last_fullsized":"2023-10-16T08:17:53.608674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:08.178740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608472+0000","last_change":"2023-10-16T08:17:51.160851+0000","last_active":"2023-10-16T08:17:53.608472+0000","last_peered":"2023-10-16T08:17:53.608472+0000","last_clean":"2023-10-16T08:17:53.608472+0000","last_became_active":"2023-10-16T08:17:51.160381+0000","last_became_peered":"2023-10-16T08:17:51.160381+0000","last_unstale":"2023-10-16T08:17:53.608472+0000","last_undegraded":"2023-10-16T08:17:53.608472+0000","last_fullsized":"2023-10-16T08:17:53.608472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:09.697131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525609+0000","last_change":"2023-10-16T08:17:51.754448+0000","last_active":"2023-10-16T08:17:53.525609+0000","last_peered":"2023-10-16T08:17:53.525609+0000","last_clean":"2023-10-16T08:17:53.525609+0000","last_became_active":"2023-10-16T08:17:51.754079+0000","last_became_peered":"2023-10-16T08:17:51.754079+0000","last_unstale":"2023-10-16T08:17:53.525609+0000","last_undegraded":"2023-10-16T08:17:53.525609+0000","last_fullsized":"2023-10-16T08:17:53.525609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:10.704012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608847+0000","last_change":"2023-10-16T08:17:51.169683+0000","last_active":"2023-10-16T08:17:53.608847+0000","last_peered":"2023-10-16T08:17:53.608847+0000","last_clean":"2023-10-16T08:17:53.608847+0000","last_became_active":"2023-10-16T08:17:51.169362+0000","last_became_peered":"2023-10-16T08:17:51.169362+0000","last_unstale":"2023-10-16T08:17:53.608847+0000","last_undegraded":"2023-10-16T08:17:53.608847+0000","last_fullsized":"2023-10-16T08:17:53.608847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:49.942117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607653+0000","last_change":"2023-10-16T08:17:51.769050+0000","last_active":"2023-10-16T08:17:53.607653+0000","last_peered":"2023-10-16T08:17:53.607653+0000","last_clean":"2023-10-16T08:17:53.607653+0000","last_became_active":"2023-10-16T08:17:51.768735+0000","last_became_peered":"2023-10-16T08:17:51.768735+0000","last_unstale":"2023-10-16T08:17:53.607653+0000","last_undegraded":"2023-10-16T08:17:53.607653+0000","last_fullsized":"2023-10-16T08:17:53.607653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:42:12.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748616+0000","last_change":"2023-10-16T08:17:51.155787+0000","last_active":"2023-10-16T08:17:52.748616+0000","last_peered":"2023-10-16T08:17:52.748616+0000","last_clean":"2023-10-16T08:17:52.748616+0000","last_became_active":"2023-10-16T08:17:51.155517+0000","last_became_peered":"2023-10-16T08:17:51.155517+0000","last_unstale":"2023-10-16T08:17:52.748616+0000","last_undegraded":"2023-10-16T08:17:52.748616+0000","last_fullsized":"2023-10-16T08:17:52.748616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:32.398724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525393+0000","last_change":"2023-10-16T08:17:51.762610+0000","last_active":"2023-10-16T08:17:53.525393+0000","last_peered":"2023-10-16T08:17:53.525393+0000","last_clean":"2023-10-16T08:17:53.525393+0000","last_became_active":"2023-10-16T08:17:51.762359+0000","last_became_peered":"2023-10-16T08:17:51.762359+0000","last_unstale":"2023-10-16T08:17:53.525393+0000","last_undegraded":"2023-10-16T08:17:53.525393+0000","last_fullsized":"2023-10-16T08:17:53.525393+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:06.072317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748629+0000","last_change":"2023-10-16T08:17:51.147628+0000","last_active":"2023-10-16T08:17:52.748629+0000","last_peered":"2023-10-16T08:17:52.748629+0000","last_clean":"2023-10-16T08:17:52.748629+0000","last_became_active":"2023-10-16T08:17:51.147364+0000","last_became_peered":"2023-10-16T08:17:51.147364+0000","last_unstale":"2023-10-16T08:17:52.748629+0000","last_undegraded":"2023-10-16T08:17:52.748629+0000","last_fullsized":"2023-10-16T08:17:52.748629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:55.966020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748413+0000","last_change":"2023-10-16T08:17:51.755564+0000","last_active":"2023-10-16T08:17:52.748413+0000","last_peered":"2023-10-16T08:17:52.748413+0000","last_clean":"2023-10-16T08:17:52.748413+0000","last_became_active":"2023-10-16T08:17:51.754643+0000","last_became_peered":"2023-10-16T08:17:51.754643+0000","last_unstale":"2023-10-16T08:17:52.748413+0000","last_undegraded":"2023-10-16T08:17:52.748413+0000","last_fullsized":"2023-10-16T08:17:52.748413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:08.178740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747560+0000","last_change":"2023-10-16T08:17:51.159148+0000","last_active":"2023-10-16T08:17:52.747560+0000","last_peered":"2023-10-16T08:17:52.747560+0000","last_clean":"2023-10-16T08:17:52.747560+0000","last_became_active":"2023-10-16T08:17:51.158847+0000","last_became_peered":"2023-10-16T08:17:51.158847+0000","last_unstale":"2023-10-16T08:17:52.747560+0000","last_undegraded":"2023-10-16T08:17:52.747560+0000","last_fullsized":"2023-10-16T08:17:52.747560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:42:11.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607968+0000","last_change":"2023-10-16T08:17:51.769837+0000","last_active":"2023-10-16T08:17:53.607968+0000","last_peered":"2023-10-16T08:17:53.607968+0000","last_clean":"2023-10-16T08:17:53.607968+0000","last_became_active":"2023-10-16T08:17:51.769570+0000","last_became_peered":"2023-10-16T08:17:51.769570+0000","last_unstale":"2023-10-16T08:17:53.607968+0000","last_undegraded":"2023-10-16T08:17:53.607968+0000","last_fullsized":"2023-10-16T08:17:53.607968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:33.405605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748968+0000","last_change":"2023-10-16T08:17:51.163789+0000","last_active":"2023-10-16T08:17:52.748968+0000","last_peered":"2023-10-16T08:17:52.748968+0000","last_clean":"2023-10-16T08:17:52.748968+0000","last_became_active":"2023-10-16T08:17:51.163577+0000","last_became_peered":"2023-10-16T08:17:51.163577+0000","last_unstale":"2023-10-16T08:17:52.748968+0000","last_undegraded":"2023-10-16T08:17:52.748968+0000","last_fullsized":"2023-10-16T08:17:52.748968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:16.834530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748646+0000","last_change":"2023-10-16T08:17:51.754705+0000","last_active":"2023-10-16T08:17:52.748646+0000","last_peered":"2023-10-16T08:17:52.748646+0000","last_clean":"2023-10-16T08:17:52.748646+0000","last_became_active":"2023-10-16T08:17:51.754369+0000","last_became_peered":"2023-10-16T08:17:51.754369+0000","last_unstale":"2023-10-16T08:17:52.748646+0000","last_undegraded":"2023-10-16T08:17:52.748646+0000","last_fullsized":"2023-10-16T08:17:52.748646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:46.903445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607307+0000","last_change":"2023-10-16T08:17:51.168902+0000","last_active":"2023-10-16T08:17:53.607307+0000","last_peered":"2023-10-16T08:17:53.607307+0000","last_clean":"2023-10-16T08:17:53.607307+0000","last_became_active":"2023-10-16T08:17:51.168615+0000","last_became_peered":"2023-10-16T08:17:51.168615+0000","last_unstale":"2023-10-16T08:17:53.607307+0000","last_undegraded":"2023-10-16T08:17:53.607307+0000","last_fullsized":"2023-10-16T08:17:53.607307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:31.259931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748645+0000","last_change":"2023-10-16T08:17:51.760020+0000","last_active":"2023-10-16T08:17:52.748645+0000","last_peered":"2023-10-16T08:17:52.748645+0000","last_clean":"2023-10-16T08:17:52.748645+0000","last_became_active":"2023-10-16T08:17:51.759750+0000","last_became_peered":"2023-10-16T08:17:51.759750+0000","last_unstale":"2023-10-16T08:17:52.748645+0000","last_undegraded":"2023-10-16T08:17:52.748645+0000","last_fullsized":"2023-10-16T08:17:52.748645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:44.565473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609439+0000","last_change":"2023-10-16T08:17:51.168352+0000","last_active":"2023-10-16T08:17:53.609439+0000","last_peered":"2023-10-16T08:17:53.609439+0000","last_clean":"2023-10-16T08:17:53.609439+0000","last_became_active":"2023-10-16T08:17:51.168066+0000","last_became_peered":"2023-10-16T08:17:51.168066+0000","last_unstale":"2023-10-16T08:17:53.609439+0000","last_undegraded":"2023-10-16T08:17:53.609439+0000","last_fullsized":"2023-10-16T08:17:53.609439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:48.504953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608759+0000","last_change":"2023-10-16T08:17:51.758972+0000","last_active":"2023-10-16T08:17:53.608759+0000","last_peered":"2023-10-16T08:17:53.608759+0000","last_clean":"2023-10-16T08:17:53.608759+0000","last_became_active":"2023-10-16T08:17:51.758652+0000","last_became_peered":"2023-10-16T08:17:51.758652+0000","last_unstale":"2023-10-16T08:17:53.608759+0000","last_undegraded":"2023-10-16T08:17:53.608759+0000","last_fullsized":"2023-10-16T08:17:53.608759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:38.450749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609003+0000","last_change":"2023-10-16T08:17:51.163599+0000","last_active":"2023-10-16T08:17:53.609003+0000","last_peered":"2023-10-16T08:17:53.609003+0000","last_clean":"2023-10-16T08:17:53.609003+0000","last_became_active":"2023-10-16T08:17:51.163175+0000","last_became_peered":"2023-10-16T08:17:51.163175+0000","last_unstale":"2023-10-16T08:17:53.609003+0000","last_undegraded":"2023-10-16T08:17:53.609003+0000","last_fullsized":"2023-10-16T08:17:53.609003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:31.259931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608781+0000","last_change":"2023-10-16T08:17:51.771158+0000","last_active":"2023-10-16T08:17:53.608781+0000","last_peered":"2023-10-16T08:17:53.608781+0000","last_clean":"2023-10-16T08:17:53.608781+0000","last_became_active":"2023-10-16T08:17:51.770813+0000","last_became_peered":"2023-10-16T08:17:51.770813+0000","last_unstale":"2023-10-16T08:17:53.608781+0000","last_undegraded":"2023-10-16T08:17:53.608781+0000","last_fullsized":"2023-10-16T08:17:53.608781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:52.230821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.606171+0000","last_change":"2023-10-16T08:17:51.162707+0000","last_active":"2023-10-16T08:17:53.606171+0000","last_peered":"2023-10-16T08:17:53.606171+0000","last_clean":"2023-10-16T08:17:53.606171+0000","last_became_active":"2023-10-16T08:17:51.160604+0000","last_became_peered":"2023-10-16T08:17:51.160604+0000","last_unstale":"2023-10-16T08:17:53.606171+0000","last_undegraded":"2023-10-16T08:17:53.606171+0000","last_fullsized":"2023-10-16T08:17:53.606171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:02.022071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605994+0000","last_change":"2023-10-16T08:17:51.763433+0000","last_active":"2023-10-16T08:17:53.605994+0000","last_peered":"2023-10-16T08:17:53.605994+0000","last_clean":"2023-10-16T08:17:53.605994+0000","last_became_active":"2023-10-16T08:17:51.763163+0000","last_became_peered":"2023-10-16T08:17:51.763163+0000","last_unstale":"2023-10-16T08:17:53.605994+0000","last_undegraded":"2023-10-16T08:17:53.605994+0000","last_fullsized":"2023-10-16T08:17:53.605994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:38.864578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747730+0000","last_change":"2023-10-16T08:17:51.166826+0000","last_active":"2023-10-16T08:17:52.747730+0000","last_peered":"2023-10-16T08:17:52.747730+0000","last_clean":"2023-10-16T08:17:52.747730+0000","last_became_active":"2023-10-16T08:17:51.166493+0000","last_became_peered":"2023-10-16T08:17:51.166493+0000","last_unstale":"2023-10-16T08:17:52.747730+0000","last_undegraded":"2023-10-16T08:17:52.747730+0000","last_fullsized":"2023-10-16T08:17:52.747730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:34.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607476+0000","last_change":"2023-10-16T08:17:51.771336+0000","last_active":"2023-10-16T08:17:53.607476+0000","last_peered":"2023-10-16T08:17:53.607476+0000","last_clean":"2023-10-16T08:17:53.607476+0000","last_became_active":"2023-10-16T08:17:51.770465+0000","last_became_peered":"2023-10-16T08:17:51.770465+0000","last_unstale":"2023-10-16T08:17:53.607476+0000","last_undegraded":"2023-10-16T08:17:53.607476+0000","last_fullsized":"2023-10-16T08:17:53.607476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:22.519364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605796+0000","last_change":"2023-10-16T08:17:51.163226+0000","last_active":"2023-10-16T08:17:53.605796+0000","last_peered":"2023-10-16T08:17:53.605796+0000","last_clean":"2023-10-16T08:17:53.605796+0000","last_became_active":"2023-10-16T08:17:51.162912+0000","last_became_peered":"2023-10-16T08:17:51.162912+0000","last_unstale":"2023-10-16T08:17:53.605796+0000","last_undegraded":"2023-10-16T08:17:53.605796+0000","last_fullsized":"2023-10-16T08:17:53.605796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:39.792343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525391+0000","last_change":"2023-10-16T08:17:51.760730+0000","last_active":"2023-10-16T08:17:53.525391+0000","last_peered":"2023-10-16T08:17:53.525391+0000","last_clean":"2023-10-16T08:17:53.525391+0000","last_became_active":"2023-10-16T08:17:51.760463+0000","last_became_peered":"2023-10-16T08:17:51.760463+0000","last_unstale":"2023-10-16T08:17:53.525391+0000","last_undegraded":"2023-10-16T08:17:53.525391+0000","last_fullsized":"2023-10-16T08:17:53.525391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:35.540947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605775+0000","last_change":"2023-10-16T08:17:51.167300+0000","last_active":"2023-10-16T08:17:53.605775+0000","last_peered":"2023-10-16T08:17:53.605775+0000","last_clean":"2023-10-16T08:17:53.605775+0000","last_became_active":"2023-10-16T08:17:51.166996+0000","last_became_peered":"2023-10-16T08:17:51.166996+0000","last_unstale":"2023-10-16T08:17:53.605775+0000","last_undegraded":"2023-10-16T08:17:53.605775+0000","last_fullsized":"2023-10-16T08:17:53.605775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:22:58.577489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609268+0000","last_change":"2023-10-16T08:17:51.753909+0000","last_active":"2023-10-16T08:17:53.609268+0000","last_peered":"2023-10-16T08:17:53.609268+0000","last_clean":"2023-10-16T08:17:53.609268+0000","last_became_active":"2023-10-16T08:17:51.753637+0000","last_became_peered":"2023-10-16T08:17:51.753637+0000","last_unstale":"2023-10-16T08:17:53.609268+0000","last_undegraded":"2023-10-16T08:17:53.609268+0000","last_fullsized":"2023-10-16T08:17:53.609268+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:46.903445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607830+0000","last_change":"2023-10-16T08:17:51.161844+0000","last_active":"2023-10-16T08:17:53.607830+0000","last_peered":"2023-10-16T08:17:53.607830+0000","last_clean":"2023-10-16T08:17:53.607830+0000","last_became_active":"2023-10-16T08:17:51.161544+0000","last_became_peered":"2023-10-16T08:17:51.161544+0000","last_unstale":"2023-10-16T08:17:53.607830+0000","last_undegraded":"2023-10-16T08:17:53.607830+0000","last_fullsized":"2023-10-16T08:17:53.607830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:04.543107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748505+0000","last_change":"2023-10-16T08:17:51.764623+0000","last_active":"2023-10-16T08:17:52.748505+0000","last_peered":"2023-10-16T08:17:52.748505+0000","last_clean":"2023-10-16T08:17:52.748505+0000","last_became_active":"2023-10-16T08:17:51.764426+0000","last_became_peered":"2023-10-16T08:17:51.764426+0000","last_unstale":"2023-10-16T08:17:52.748505+0000","last_undegraded":"2023-10-16T08:17:52.748505+0000","last_fullsized":"2023-10-16T08:17:52.748505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:57.348350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605824+0000","last_change":"2023-10-16T08:17:51.159106+0000","last_active":"2023-10-16T08:17:53.605824+0000","last_peered":"2023-10-16T08:17:53.605824+0000","last_clean":"2023-10-16T08:17:53.605824+0000","last_became_active":"2023-10-16T08:17:51.158803+0000","last_became_peered":"2023-10-16T08:17:51.158803+0000","last_unstale":"2023-10-16T08:17:53.605824+0000","last_undegraded":"2023-10-16T08:17:53.605824+0000","last_fullsized":"2023-10-16T08:17:53.605824+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:33.860746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747750+0000","last_change":"2023-10-16T08:17:51.757820+0000","last_active":"2023-10-16T08:17:52.747750+0000","last_peered":"2023-10-16T08:17:52.747750+0000","last_clean":"2023-10-16T08:17:52.747750+0000","last_became_active":"2023-10-16T08:17:51.757536+0000","last_became_peered":"2023-10-16T08:17:51.757536+0000","last_unstale":"2023-10-16T08:17:52.747750+0000","last_undegraded":"2023-10-16T08:17:52.747750+0000","last_fullsized":"2023-10-16T08:17:52.747750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:50.948998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609508+0000","last_change":"2023-10-16T08:17:51.167812+0000","last_active":"2023-10-16T08:17:53.609508+0000","last_peered":"2023-10-16T08:17:53.609508+0000","last_clean":"2023-10-16T08:17:53.609508+0000","last_became_active":"2023-10-16T08:17:51.167544+0000","last_became_peered":"2023-10-16T08:17:51.167544+0000","last_unstale":"2023-10-16T08:17:53.609508+0000","last_undegraded":"2023-10-16T08:17:53.609508+0000","last_fullsized":"2023-10-16T08:17:53.609508+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:21.512482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608756+0000","last_change":"2023-10-16T08:17:51.756646+0000","last_active":"2023-10-16T08:17:53.608756+0000","last_peered":"2023-10-16T08:17:53.608756+0000","last_clean":"2023-10-16T08:17:53.608756+0000","last_became_active":"2023-10-16T08:17:51.756370+0000","last_became_peered":"2023-10-16T08:17:51.756370+0000","last_unstale":"2023-10-16T08:17:53.608756+0000","last_undegraded":"2023-10-16T08:17:53.608756+0000","last_fullsized":"2023-10-16T08:17:53.608756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:38.450749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605897+0000","last_change":"2023-10-16T08:17:51.167361+0000","last_active":"2023-10-16T08:17:53.605897+0000","last_peered":"2023-10-16T08:17:53.605897+0000","last_clean":"2023-10-16T08:17:53.605897+0000","last_became_active":"2023-10-16T08:17:51.167086+0000","last_became_peered":"2023-10-16T08:17:51.167086+0000","last_unstale":"2023-10-16T08:17:53.605897+0000","last_undegraded":"2023-10-16T08:17:53.605897+0000","last_fullsized":"2023-10-16T08:17:53.605897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:51.087792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748801+0000","last_change":"2023-10-16T08:17:51.770518+0000","last_active":"2023-10-16T08:17:52.748801+0000","last_peered":"2023-10-16T08:17:52.748801+0000","last_clean":"2023-10-16T08:17:52.748801+0000","last_became_active":"2023-10-16T08:17:51.770247+0000","last_became_peered":"2023-10-16T08:17:51.770247+0000","last_unstale":"2023-10-16T08:17:52.748801+0000","last_undegraded":"2023-10-16T08:17:52.748801+0000","last_fullsized":"2023-10-16T08:17:52.748801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:10.208587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525477+0000","last_change":"2023-10-16T08:17:51.155822+0000","last_active":"2023-10-16T08:17:53.525477+0000","last_peered":"2023-10-16T08:17:53.525477+0000","last_clean":"2023-10-16T08:17:53.525477+0000","last_became_active":"2023-10-16T08:17:51.155433+0000","last_became_peered":"2023-10-16T08:17:51.155433+0000","last_unstale":"2023-10-16T08:17:53.525477+0000","last_undegraded":"2023-10-16T08:17:53.525477+0000","last_fullsized":"2023-10-16T08:17:53.525477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:45.202816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525575+0000","last_change":"2023-10-16T08:17:51.754218+0000","last_active":"2023-10-16T08:17:53.525575+0000","last_peered":"2023-10-16T08:17:53.525575+0000","last_clean":"2023-10-16T08:17:53.525575+0000","last_became_active":"2023-10-16T08:17:51.753947+0000","last_became_peered":"2023-10-16T08:17:51.753947+0000","last_unstale":"2023-10-16T08:17:53.525575+0000","last_undegraded":"2023-10-16T08:17:53.525575+0000","last_fullsized":"2023-10-16T08:17:53.525575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:50.948998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525602+0000","last_change":"2023-10-16T08:17:51.149101+0000","last_active":"2023-10-16T08:17:53.525602+0000","last_peered":"2023-10-16T08:17:53.525602+0000","last_clean":"2023-10-16T08:17:53.525602+0000","last_became_active":"2023-10-16T08:17:51.148619+0000","last_became_peered":"2023-10-16T08:17:51.148619+0000","last_unstale":"2023-10-16T08:17:53.525602+0000","last_undegraded":"2023-10-16T08:17:53.525602+0000","last_fullsized":"2023-10-16T08:17:53.525602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:33.860746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747499+0000","last_change":"2023-10-16T08:17:51.752263+0000","last_active":"2023-10-16T08:17:52.747499+0000","last_peered":"2023-10-16T08:17:52.747499+0000","last_clean":"2023-10-16T08:17:52.747499+0000","last_became_active":"2023-10-16T08:17:51.751928+0000","last_became_peered":"2023-10-16T08:17:51.751928+0000","last_unstale":"2023-10-16T08:17:52.747499+0000","last_undegraded":"2023-10-16T08:17:52.747499+0000","last_fullsized":"2023-10-16T08:17:52.747499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:16.234591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749180+0000","last_change":"2023-10-16T08:17:51.156434+0000","last_active":"2023-10-16T08:17:52.749180+0000","last_peered":"2023-10-16T08:17:52.749180+0000","last_clean":"2023-10-16T08:17:52.749180+0000","last_became_active":"2023-10-16T08:17:51.156200+0000","last_became_peered":"2023-10-16T08:17:51.156200+0000","last_unstale":"2023-10-16T08:17:52.749180+0000","last_undegraded":"2023-10-16T08:17:52.749180+0000","last_fullsized":"2023-10-16T08:17:52.749180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:39.792343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607834+0000","last_change":"2023-10-16T08:17:51.767879+0000","last_active":"2023-10-16T08:17:53.607834+0000","last_peered":"2023-10-16T08:17:53.607834+0000","last_clean":"2023-10-16T08:17:53.607834+0000","last_became_active":"2023-10-16T08:17:51.767601+0000","last_became_peered":"2023-10-16T08:17:51.767601+0000","last_unstale":"2023-10-16T08:17:53.607834+0000","last_undegraded":"2023-10-16T08:17:53.607834+0000","last_fullsized":"2023-10-16T08:17:53.607834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:06.072317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607618+0000","last_change":"2023-10-16T08:17:51.163800+0000","last_active":"2023-10-16T08:17:53.607618+0000","last_peered":"2023-10-16T08:17:53.607618+0000","last_clean":"2023-10-16T08:17:53.607618+0000","last_became_active":"2023-10-16T08:17:51.163229+0000","last_became_peered":"2023-10-16T08:17:51.163229+0000","last_unstale":"2023-10-16T08:17:53.607618+0000","last_undegraded":"2023-10-16T08:17:53.607618+0000","last_fullsized":"2023-10-16T08:17:53.607618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:04.543107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608571+0000","last_change":"2023-10-16T08:17:51.755359+0000","last_active":"2023-10-16T08:17:53.608571+0000","last_peered":"2023-10-16T08:17:53.608571+0000","last_clean":"2023-10-16T08:17:53.608571+0000","last_became_active":"2023-10-16T08:17:51.755060+0000","last_became_peered":"2023-10-16T08:17:51.755060+0000","last_unstale":"2023-10-16T08:17:53.608571+0000","last_undegraded":"2023-10-16T08:17:53.608571+0000","last_fullsized":"2023-10-16T08:17:53.608571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:30.638593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748319+0000","last_change":"2023-10-16T08:17:51.155157+0000","last_active":"2023-10-16T08:17:52.748319+0000","last_peered":"2023-10-16T08:17:52.748319+0000","last_clean":"2023-10-16T08:17:52.748319+0000","last_became_active":"2023-10-16T08:17:51.154881+0000","last_became_peered":"2023-10-16T08:17:51.154881+0000","last_unstale":"2023-10-16T08:17:52.748319+0000","last_undegraded":"2023-10-16T08:17:52.748319+0000","last_fullsized":"2023-10-16T08:17:52.748319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:17.067875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608631+0000","last_change":"2023-10-16T08:17:51.758422+0000","last_active":"2023-10-16T08:17:53.608631+0000","last_peered":"2023-10-16T08:17:53.608631+0000","last_clean":"2023-10-16T08:17:53.608631+0000","last_became_active":"2023-10-16T08:17:51.758162+0000","last_became_peered":"2023-10-16T08:17:51.758162+0000","last_unstale":"2023-10-16T08:17:53.608631+0000","last_undegraded":"2023-10-16T08:17:53.608631+0000","last_fullsized":"2023-10-16T08:17:53.608631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:33.756951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608685+0000","last_change":"2023-10-16T08:17:51.164292+0000","last_active":"2023-10-16T08:17:53.608685+0000","last_peered":"2023-10-16T08:17:53.608685+0000","last_clean":"2023-10-16T08:17:53.608685+0000","last_became_active":"2023-10-16T08:17:51.164013+0000","last_became_peered":"2023-10-16T08:17:51.164013+0000","last_unstale":"2023-10-16T08:17:53.608685+0000","last_undegraded":"2023-10-16T08:17:53.608685+0000","last_fullsized":"2023-10-16T08:17:53.608685+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:15.227710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-16T08:17:51.745273+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:17:51.745273+0000","last_peered":"2023-10-16T08:17:51.745273+0000","last_clean":"2023-10-16T08:17:51.745273+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:17:51.745273+0000","last_undegraded":"2023-10-16T08:17:51.745273+0000","last_fullsized":"2023-10-16T08:17:51.745273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":779324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.748}]},{"osd":4,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.628}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82899999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66300000000000003}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.76500000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.78600000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.79800000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.62},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81799999999999995}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":187878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.628},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56899999999999995}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.41099999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.44500000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":195565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.96099999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64300000000000002}]},{"osd":6,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.93500000000000005}]},{"osd":7,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.91100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":187344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.2779999999999996,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":4.8550000000000004,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.399,"5min":4.399,"15min":4.399},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.8220000000000001,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.45900000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.495},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.5579999999999998,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.624}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.5709999999999997,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.7519999999999998,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":194009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.73099999999999998}]},{"osd":5,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":777676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.626}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58699999999999997}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84199999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.94499999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0329999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:17:56.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:17:56.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d6b65640 1 Processor -- start 2023-10-16T08:17:56.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d6b65640 1 -- start start 2023-10-16T08:17:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d6b65640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39d00d3010 0x7f39d00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d6b65640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d00ce770 con 0x7f39d015cb90 2023-10-16T08:17:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d6b65640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d00d0600 con 0x7f39d00d15d0 2023-10-16T08:17:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d6b65640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00c6450 con 0x7f39d00d3010 2023-10-16T08:17:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d4901640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39d015cb90 0x7f39d00d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60798/0 (socket says 172.21.15.32:60798) 2023-10-16T08:17:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d4901640 1 -- 172.21.15.32:0/3994235131 learned_addr learned my addr 172.21.15.32:0/3994235131 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d5102640 1 --2- 172.21.15.32:0/3994235131 >> v2:172.21.15.32:3300/0 conn(0x7f39d00d3010 0x7f39d00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39cf7fe640 1 -- 172.21.15.32:0/3994235131 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172634840 0 0) 0x7f39d00ce770 con 0x7f39d015cb90 2023-10-16T08:17:56.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d5102640 1 -- 172.21.15.32:0/3994235131 >> v1:172.21.15.186:6789/0 conn(0x7f39d00d15d0 legacy=0x7f39d00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d5102640 1 -- 172.21.15.32:0/3994235131 >> v1:172.21.15.32:6789/0 conn(0x7f39d015cb90 legacy=0x7f39d00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.453+0000 7f39d5102640 1 -- 172.21.15.32:0/3994235131 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00d08a0 con 0x7f39d00d3010 2023-10-16T08:17:56.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d5102640 1 --2- 172.21.15.32:0/3994235131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d00cfb10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009d90 tx=0x7f39c801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376278b3663b7a7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39cf7fe640 1 -- 172.21.15.32:0/3994235131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c80a5950 con 0x7f39d00d3010 2023-10-16T08:17:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39cf7fe640 1 -- 172.21.15.32:0/3994235131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c80a5ab0 con 0x7f39d00d3010 2023-10-16T08:17:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39cf7fe640 1 -- 172.21.15.32:0/3994235131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c80a5e40 con 0x7f39d00d3010 2023-10-16T08:17:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 -- 172.21.15.32:0/3994235131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 msgr2=0x7f39d00cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 --2- 172.21.15.32:0/3994235131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d00cfb10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009d90 tx=0x7f39c801c890 comp rx=0 tx=0).stop 2023-10-16T08:17:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 -- 172.21.15.32:0/3994235131 shutdown_connections 2023-10-16T08:17:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 --2- 172.21.15.32:0/3994235131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d00cfb10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 -- 172.21.15.32:0/3994235131 >> 172.21.15.32:0/3994235131 conn(0x7f39d009bc50 msgr2=0x7f39d00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 -- 172.21.15.32:0/3994235131 shutdown_connections 2023-10-16T08:17:56.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.457+0000 7f39d6b65640 1 -- 172.21.15.32:0/3994235131 wait complete. 2023-10-16T08:17:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 Processor -- start 2023-10-16T08:17:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 -- start start 2023-10-16T08:17:56.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d00d15d0 0x7f39d0153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39d015cb90 0x7f39d006be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d01573f0 con 0x7f39d00d3010 2023-10-16T08:17:56.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d015a670 con 0x7f39d00d15d0 2023-10-16T08:17:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d015a950 con 0x7f39d015cb90 2023-10-16T08:17:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d006b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50796/0 (socket says 172.21.15.32:50796) 2023-10-16T08:17:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 -- 172.21.15.32:0/1654243613 learned_addr learned my addr 172.21.15.32:0/1654243613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:56.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d4901640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d00d15d0 0x7f39d0153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d5102640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39d015cb90 0x7f39d006be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 -- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39d015cb90 msgr2=0x7f39d006be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39d015cb90 0x7f39d006be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 -- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d00d15d0 msgr2=0x7f39d0153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d00d15d0 0x7f39d0153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 -- 172.21.15.32:0/1654243613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8009a70 con 0x7f39d00d3010 2023-10-16T08:17:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d4901640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d00d15d0 0x7f39d0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cffff640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d006b8f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c420 tx=0x7f39bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0052d0 con 0x7f39d00d3010 2023-10-16T08:17:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc005430 con 0x7f39d00d3010 2023-10-16T08:17:56.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d015ac30 con 0x7f39d00d3010 2023-10-16T08:17:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc007260 con 0x7f39d00d3010 2023-10-16T08:17:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.461+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d015b1f0 con 0x7f39d00d3010 2023-10-16T08:17:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.465+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d01f5860 con 0x7f39d00d3010 2023-10-16T08:17:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.465+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39bc0073c0 con 0x7f39d00d3010 2023-10-16T08:17:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.465+0000 7f39cd7fa640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39b40664a0 0x7f39b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.465+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f39bc0833f0 con 0x7f39d00d3010 2023-10-16T08:17:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.465+0000 7f39d4901640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39b40664a0 0x7f39b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.465+0000 7f39d4901640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39b40664a0 0x7f39b4068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39c4000c10 tx=0x7f39c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.469+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39bc050710 con 0x7f39d00d3010 2023-10-16T08:17:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.621+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39d015aac0 con 0x7f39b40664a0 2023-10-16T08:17:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.629+0000 7f39cd7fa640 1 -- 172.21.15.32:0/1654243613 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289238 (secure 0 0 0) 0x7f39d015aac0 con 0x7f39b40664a0 2023-10-16T08:17:56.640 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:17:56.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.633+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39b40664a0 msgr2=0x7f39b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.633+0000 7f39d6b65640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39b40664a0 0x7f39b4068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39c4000c10 tx=0x7f39c4009040 comp rx=0 tx=0).stop 2023-10-16T08:17:56.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.633+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 msgr2=0x7f39d006b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.633+0000 7f39d6b65640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d006b8f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c420 tx=0x7f39bc00c8e0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 shutdown_connections 2023-10-16T08:17:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39b40664a0 0x7f39b4068960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39d015cb90 0x7f39d006be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39d00d3010 0x7f39d006b8f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 --2- 172.21.15.32:0/1654243613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d00d15d0 0x7f39d0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 >> 172.21.15.32:0/1654243613 conn(0x7f39d009bc50 msgr2=0x7f39d00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 shutdown_connections 2023-10-16T08:17:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.637+0000 7f39d6b65640 1 -- 172.21.15.32:0/1654243613 wait complete. 2023-10-16T08:17:56.661 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-10-16T08:17:56.037559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233372,"kb_used_data":4696,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741348,"statfs":{"total":773094113280,"available":772855140352,"internally_reserved":0,"allocated":4808704,"data_stored":2710328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":234016315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.903558"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748795+0000","last_change":"2023-10-16T08:17:51.761871+0000","last_active":"2023-10-16T08:17:52.748795+0000","last_peered":"2023-10-16T08:17:52.748795+0000","last_clean":"2023-10-16T08:17:52.748795+0000","last_became_active":"2023-10-16T08:17:51.761568+0000","last_became_peered":"2023-10-16T08:17:51.761568+0000","last_unstale":"2023-10-16T08:17:52.748795+0000","last_undegraded":"2023-10-16T08:17:52.748795+0000","last_fullsized":"2023-10-16T08:17:52.748795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:25:30.639378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748694+0000","last_change":"2023-10-16T08:17:51.765816+0000","last_active":"2023-10-16T08:17:52.748694+0000","last_peered":"2023-10-16T08:17:52.748694+0000","last_clean":"2023-10-16T08:17:52.748694+0000","last_became_active":"2023-10-16T08:17:51.765431+0000","last_became_peered":"2023-10-16T08:17:51.765431+0000","last_unstale":"2023-10-16T08:17:52.748694+0000","last_undegraded":"2023-10-16T08:17:52.748694+0000","last_fullsized":"2023-10-16T08:17:52.748694+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:09.245608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525493+0000","last_change":"2023-10-16T08:17:51.763403+0000","last_active":"2023-10-16T08:17:53.525493+0000","last_peered":"2023-10-16T08:17:53.525493+0000","last_clean":"2023-10-16T08:17:53.525493+0000","last_became_active":"2023-10-16T08:17:51.756500+0000","last_became_peered":"2023-10-16T08:17:51.756500+0000","last_unstale":"2023-10-16T08:17:53.525493+0000","last_undegraded":"2023-10-16T08:17:53.525493+0000","last_fullsized":"2023-10-16T08:17:53.525493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:16.234591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605971+0000","last_change":"2023-10-16T08:17:51.771620+0000","last_active":"2023-10-16T08:17:53.605971+0000","last_peered":"2023-10-16T08:17:53.605971+0000","last_clean":"2023-10-16T08:17:53.605971+0000","last_became_active":"2023-10-16T08:17:51.771350+0000","last_became_peered":"2023-10-16T08:17:51.771350+0000","last_unstale":"2023-10-16T08:17:53.605971+0000","last_undegraded":"2023-10-16T08:17:53.605971+0000","last_fullsized":"2023-10-16T08:17:53.605971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:05.549988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605730+0000","last_change":"2023-10-16T08:17:51.770594+0000","last_active":"2023-10-16T08:17:53.605730+0000","last_peered":"2023-10-16T08:17:53.605730+0000","last_clean":"2023-10-16T08:17:53.605730+0000","last_became_active":"2023-10-16T08:17:51.770322+0000","last_became_peered":"2023-10-16T08:17:51.770322+0000","last_unstale":"2023-10-16T08:17:53.605730+0000","last_undegraded":"2023-10-16T08:17:53.605730+0000","last_fullsized":"2023-10-16T08:17:53.605730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:02:28.473257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747574+0000","last_change":"2023-10-16T08:17:51.752780+0000","last_active":"2023-10-16T08:17:52.747574+0000","last_peered":"2023-10-16T08:17:52.747574+0000","last_clean":"2023-10-16T08:17:52.747574+0000","last_became_active":"2023-10-16T08:17:51.752519+0000","last_became_peered":"2023-10-16T08:17:51.752519+0000","last_unstale":"2023-10-16T08:17:52.747574+0000","last_undegraded":"2023-10-16T08:17:52.747574+0000","last_fullsized":"2023-10-16T08:17:52.747574+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:52.230821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608026+0000","last_change":"2023-10-16T08:17:51.760424+0000","last_active":"2023-10-16T08:17:53.608026+0000","last_peered":"2023-10-16T08:17:53.608026+0000","last_clean":"2023-10-16T08:17:53.608026+0000","last_became_active":"2023-10-16T08:17:51.758777+0000","last_became_peered":"2023-10-16T08:17:51.758777+0000","last_unstale":"2023-10-16T08:17:53.608026+0000","last_undegraded":"2023-10-16T08:17:53.608026+0000","last_fullsized":"2023-10-16T08:17:53.608026+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:56.972901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748274+0000","last_change":"2023-10-16T08:17:51.767472+0000","last_active":"2023-10-16T08:17:52.748274+0000","last_peered":"2023-10-16T08:17:52.748274+0000","last_clean":"2023-10-16T08:17:52.748274+0000","last_became_active":"2023-10-16T08:17:51.767135+0000","last_became_peered":"2023-10-16T08:17:51.767135+0000","last_unstale":"2023-10-16T08:17:52.748274+0000","last_undegraded":"2023-10-16T08:17:52.748274+0000","last_fullsized":"2023-10-16T08:17:52.748274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:30.638593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525747+0000","last_change":"2023-10-16T08:17:51.756513+0000","last_active":"2023-10-16T08:17:53.525747+0000","last_peered":"2023-10-16T08:17:53.525747+0000","last_clean":"2023-10-16T08:17:53.525747+0000","last_became_active":"2023-10-16T08:17:51.756269+0000","last_became_peered":"2023-10-16T08:17:51.756269+0000","last_unstale":"2023-10-16T08:17:53.525747+0000","last_undegraded":"2023-10-16T08:17:53.525747+0000","last_fullsized":"2023-10-16T08:17:53.525747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:49.511834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608376+0000","last_change":"2023-10-16T08:17:51.765803+0000","last_active":"2023-10-16T08:17:53.608376+0000","last_peered":"2023-10-16T08:17:53.608376+0000","last_clean":"2023-10-16T08:17:53.608376+0000","last_became_active":"2023-10-16T08:17:51.765526+0000","last_became_peered":"2023-10-16T08:17:51.765526+0000","last_unstale":"2023-10-16T08:17:53.608376+0000","last_undegraded":"2023-10-16T08:17:53.608376+0000","last_fullsized":"2023-10-16T08:17:53.608376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:18.074756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608631+0000","last_change":"2023-10-16T08:17:51.767369+0000","last_active":"2023-10-16T08:17:53.608631+0000","last_peered":"2023-10-16T08:17:53.608631+0000","last_clean":"2023-10-16T08:17:53.608631+0000","last_became_active":"2023-10-16T08:17:51.767099+0000","last_became_peered":"2023-10-16T08:17:51.767099+0000","last_unstale":"2023-10-16T08:17:53.608631+0000","last_undegraded":"2023-10-16T08:17:53.608631+0000","last_fullsized":"2023-10-16T08:17:53.608631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:30.638593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525459+0000","last_change":"2023-10-16T08:17:51.757496+0000","last_active":"2023-10-16T08:17:53.525459+0000","last_peered":"2023-10-16T08:17:53.525459+0000","last_clean":"2023-10-16T08:17:53.525459+0000","last_became_active":"2023-10-16T08:17:51.756308+0000","last_became_peered":"2023-10-16T08:17:51.756308+0000","last_unstale":"2023-10-16T08:17:53.525459+0000","last_undegraded":"2023-10-16T08:17:53.525459+0000","last_fullsized":"2023-10-16T08:17:53.525459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:11.855807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609267+0000","last_change":"2023-10-16T08:17:51.766543+0000","last_active":"2023-10-16T08:17:53.609267+0000","last_peered":"2023-10-16T08:17:53.609267+0000","last_clean":"2023-10-16T08:17:53.609267+0000","last_became_active":"2023-10-16T08:17:51.766177+0000","last_became_peered":"2023-10-16T08:17:51.766177+0000","last_unstale":"2023-10-16T08:17:53.609267+0000","last_undegraded":"2023-10-16T08:17:53.609267+0000","last_fullsized":"2023-10-16T08:17:53.609267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:44.565473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607655+0000","last_change":"2023-10-16T08:17:51.771095+0000","last_active":"2023-10-16T08:17:53.607655+0000","last_peered":"2023-10-16T08:17:53.607655+0000","last_clean":"2023-10-16T08:17:53.607655+0000","last_became_active":"2023-10-16T08:17:51.770786+0000","last_became_peered":"2023-10-16T08:17:51.770786+0000","last_unstale":"2023-10-16T08:17:53.607655+0000","last_undegraded":"2023-10-16T08:17:53.607655+0000","last_fullsized":"2023-10-16T08:17:53.607655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:35.540947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608715+0000","last_change":"2023-10-16T08:17:51.766213+0000","last_active":"2023-10-16T08:17:53.608715+0000","last_peered":"2023-10-16T08:17:53.608715+0000","last_clean":"2023-10-16T08:17:53.608715+0000","last_became_active":"2023-10-16T08:17:51.765857+0000","last_became_peered":"2023-10-16T08:17:51.765857+0000","last_unstale":"2023-10-16T08:17:53.608715+0000","last_undegraded":"2023-10-16T08:17:53.608715+0000","last_fullsized":"2023-10-16T08:17:53.608715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:08.178740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607307+0000","last_change":"2023-10-16T08:17:51.760331+0000","last_active":"2023-10-16T08:17:53.607307+0000","last_peered":"2023-10-16T08:17:53.607307+0000","last_clean":"2023-10-16T08:17:53.607307+0000","last_became_active":"2023-10-16T08:17:51.760013+0000","last_became_peered":"2023-10-16T08:17:51.760013+0000","last_unstale":"2023-10-16T08:17:53.607307+0000","last_undegraded":"2023-10-16T08:17:53.607307+0000","last_fullsized":"2023-10-16T08:17:53.607307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:10.704012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608456+0000","last_change":"2023-10-16T08:17:51.757160+0000","last_active":"2023-10-16T08:17:53.608456+0000","last_peered":"2023-10-16T08:17:53.608456+0000","last_clean":"2023-10-16T08:17:53.608456+0000","last_became_active":"2023-10-16T08:17:51.756344+0000","last_became_peered":"2023-10-16T08:17:51.756344+0000","last_unstale":"2023-10-16T08:17:53.608456+0000","last_undegraded":"2023-10-16T08:17:53.608456+0000","last_fullsized":"2023-10-16T08:17:53.608456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:18.074756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749338+0000","last_change":"2023-10-16T08:17:51.773910+0000","last_active":"2023-10-16T08:17:52.749338+0000","last_peered":"2023-10-16T08:17:52.749338+0000","last_clean":"2023-10-16T08:17:52.749338+0000","last_became_active":"2023-10-16T08:17:51.773592+0000","last_became_peered":"2023-10-16T08:17:51.773592+0000","last_unstale":"2023-10-16T08:17:52.749338+0000","last_undegraded":"2023-10-16T08:17:52.749338+0000","last_fullsized":"2023-10-16T08:17:52.749338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:25.146020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.606109+0000","last_change":"2023-10-16T08:17:51.762446+0000","last_active":"2023-10-16T08:17:53.606109+0000","last_peered":"2023-10-16T08:17:53.606109+0000","last_clean":"2023-10-16T08:17:53.606109+0000","last_became_active":"2023-10-16T08:17:51.762174+0000","last_became_peered":"2023-10-16T08:17:51.762174+0000","last_unstale":"2023-10-16T08:17:53.606109+0000","last_undegraded":"2023-10-16T08:17:53.606109+0000","last_fullsized":"2023-10-16T08:17:53.606109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:49.836706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749295+0000","last_change":"2023-10-16T08:17:51.772967+0000","last_active":"2023-10-16T08:17:52.749295+0000","last_peered":"2023-10-16T08:17:52.749295+0000","last_clean":"2023-10-16T08:17:52.749295+0000","last_became_active":"2023-10-16T08:17:51.772195+0000","last_became_peered":"2023-10-16T08:17:51.772195+0000","last_unstale":"2023-10-16T08:17:52.749295+0000","last_undegraded":"2023-10-16T08:17:52.749295+0000","last_fullsized":"2023-10-16T08:17:52.749295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:49.836706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609392+0000","last_change":"2023-10-16T08:17:51.754447+0000","last_active":"2023-10-16T08:17:53.609392+0000","last_peered":"2023-10-16T08:17:53.609392+0000","last_clean":"2023-10-16T08:17:53.609392+0000","last_became_active":"2023-10-16T08:17:51.754098+0000","last_became_peered":"2023-10-16T08:17:51.754098+0000","last_unstale":"2023-10-16T08:17:53.609392+0000","last_undegraded":"2023-10-16T08:17:53.609392+0000","last_fullsized":"2023-10-16T08:17:53.609392+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:09.245608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608032+0000","last_change":"2023-10-16T08:17:51.771524+0000","last_active":"2023-10-16T08:17:53.608032+0000","last_peered":"2023-10-16T08:17:53.608032+0000","last_clean":"2023-10-16T08:17:53.608032+0000","last_became_active":"2023-10-16T08:17:51.770806+0000","last_became_peered":"2023-10-16T08:17:51.770806+0000","last_unstale":"2023-10-16T08:17:53.608032+0000","last_undegraded":"2023-10-16T08:17:53.608032+0000","last_fullsized":"2023-10-16T08:17:53.608032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:11.855807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607437+0000","last_change":"2023-10-16T08:17:51.765141+0000","last_active":"2023-10-16T08:17:53.607437+0000","last_peered":"2023-10-16T08:17:53.607437+0000","last_clean":"2023-10-16T08:17:53.607437+0000","last_became_active":"2023-10-16T08:17:51.764755+0000","last_became_peered":"2023-10-16T08:17:51.764755+0000","last_unstale":"2023-10-16T08:17:53.607437+0000","last_undegraded":"2023-10-16T08:17:53.607437+0000","last_fullsized":"2023-10-16T08:17:53.607437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:49.511834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605752+0000","last_change":"2023-10-16T08:17:51.768598+0000","last_active":"2023-10-16T08:17:53.605752+0000","last_peered":"2023-10-16T08:17:53.605752+0000","last_clean":"2023-10-16T08:17:53.605752+0000","last_became_active":"2023-10-16T08:17:51.768382+0000","last_became_peered":"2023-10-16T08:17:51.768382+0000","last_unstale":"2023-10-16T08:17:53.605752+0000","last_undegraded":"2023-10-16T08:17:53.605752+0000","last_fullsized":"2023-10-16T08:17:53.605752+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:46.209697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749505+0000","last_change":"2023-10-16T08:17:51.760433+0000","last_active":"2023-10-16T08:17:52.749505+0000","last_peered":"2023-10-16T08:17:52.749505+0000","last_clean":"2023-10-16T08:17:52.749505+0000","last_became_active":"2023-10-16T08:17:51.760140+0000","last_became_peered":"2023-10-16T08:17:51.760140+0000","last_unstale":"2023-10-16T08:17:52.749505+0000","last_undegraded":"2023-10-16T08:17:52.749505+0000","last_fullsized":"2023-10-16T08:17:52.749505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:35.540947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747917+0000","last_change":"2023-10-16T08:17:51.760272+0000","last_active":"2023-10-16T08:17:52.747917+0000","last_peered":"2023-10-16T08:17:52.747917+0000","last_clean":"2023-10-16T08:17:52.747917+0000","last_became_active":"2023-10-16T08:17:51.758030+0000","last_became_peered":"2023-10-16T08:17:51.758030+0000","last_unstale":"2023-10-16T08:17:52.747917+0000","last_undegraded":"2023-10-16T08:17:52.747917+0000","last_fullsized":"2023-10-16T08:17:52.747917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:49.511834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749372+0000","last_change":"2023-10-16T08:17:51.764293+0000","last_active":"2023-10-16T08:17:52.749372+0000","last_peered":"2023-10-16T08:17:52.749372+0000","last_clean":"2023-10-16T08:17:52.749372+0000","last_became_active":"2023-10-16T08:17:51.763961+0000","last_became_peered":"2023-10-16T08:17:51.763961+0000","last_unstale":"2023-10-16T08:17:52.749372+0000","last_undegraded":"2023-10-16T08:17:52.749372+0000","last_fullsized":"2023-10-16T08:17:52.749372+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:42:12.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608260+0000","last_change":"2023-10-16T08:17:51.763333+0000","last_active":"2023-10-16T08:17:53.608260+0000","last_peered":"2023-10-16T08:17:53.608260+0000","last_clean":"2023-10-16T08:17:53.608260+0000","last_became_active":"2023-10-16T08:17:51.762447+0000","last_became_peered":"2023-10-16T08:17:51.762447+0000","last_unstale":"2023-10-16T08:17:53.608260+0000","last_undegraded":"2023-10-16T08:17:53.608260+0000","last_fullsized":"2023-10-16T08:17:53.608260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:33.405605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749066+0000","last_change":"2023-10-16T08:17:51.752557+0000","last_active":"2023-10-16T08:17:52.749066+0000","last_peered":"2023-10-16T08:17:52.749066+0000","last_clean":"2023-10-16T08:17:52.749066+0000","last_became_active":"2023-10-16T08:17:51.752286+0000","last_became_peered":"2023-10-16T08:17:51.752286+0000","last_unstale":"2023-10-16T08:17:52.749066+0000","last_undegraded":"2023-10-16T08:17:52.749066+0000","last_fullsized":"2023-10-16T08:17:52.749066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:05.549988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749172+0000","last_change":"2023-10-16T08:17:51.753662+0000","last_active":"2023-10-16T08:17:52.749172+0000","last_peered":"2023-10-16T08:17:52.749172+0000","last_clean":"2023-10-16T08:17:52.749172+0000","last_became_active":"2023-10-16T08:17:51.753380+0000","last_became_peered":"2023-10-16T08:17:51.753380+0000","last_unstale":"2023-10-16T08:17:52.749172+0000","last_undegraded":"2023-10-16T08:17:52.749172+0000","last_fullsized":"2023-10-16T08:17:52.749172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:38.864578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607858+0000","last_change":"2023-10-16T08:17:51.769094+0000","last_active":"2023-10-16T08:17:53.607858+0000","last_peered":"2023-10-16T08:17:53.607858+0000","last_clean":"2023-10-16T08:17:53.607858+0000","last_became_active":"2023-10-16T08:17:51.768850+0000","last_became_peered":"2023-10-16T08:17:51.768850+0000","last_unstale":"2023-10-16T08:17:53.607858+0000","last_undegraded":"2023-10-16T08:17:53.607858+0000","last_fullsized":"2023-10-16T08:17:53.607858+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:56.972901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747229+0000","last_change":"2023-10-16T08:17:51.764687+0000","last_active":"2023-10-16T08:17:52.747229+0000","last_peered":"2023-10-16T08:17:52.747229+0000","last_clean":"2023-10-16T08:17:52.747229+0000","last_became_active":"2023-10-16T08:17:51.764386+0000","last_became_peered":"2023-10-16T08:17:51.764386+0000","last_unstale":"2023-10-16T08:17:52.747229+0000","last_undegraded":"2023-10-16T08:17:52.747229+0000","last_fullsized":"2023-10-16T08:17:52.747229+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:11.855807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608541+0000","last_change":"2023-10-16T08:17:51.764566+0000","last_active":"2023-10-16T08:17:53.608541+0000","last_peered":"2023-10-16T08:17:53.608541+0000","last_clean":"2023-10-16T08:17:53.608541+0000","last_became_active":"2023-10-16T08:17:51.763026+0000","last_became_peered":"2023-10-16T08:17:51.763026+0000","last_unstale":"2023-10-16T08:17:53.608541+0000","last_undegraded":"2023-10-16T08:17:53.608541+0000","last_fullsized":"2023-10-16T08:17:53.608541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:33.756951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749545+0000","last_change":"2023-10-16T08:17:51.166289+0000","last_active":"2023-10-16T08:17:52.749545+0000","last_peered":"2023-10-16T08:17:52.749545+0000","last_clean":"2023-10-16T08:17:52.749545+0000","last_became_active":"2023-10-16T08:17:51.166015+0000","last_became_peered":"2023-10-16T08:17:51.166015+0000","last_unstale":"2023-10-16T08:17:52.749545+0000","last_undegraded":"2023-10-16T08:17:52.749545+0000","last_fullsized":"2023-10-16T08:17:52.749545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:45.202816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747874+0000","last_change":"2023-10-16T08:17:51.755280+0000","last_active":"2023-10-16T08:17:52.747874+0000","last_peered":"2023-10-16T08:17:52.747874+0000","last_clean":"2023-10-16T08:17:52.747874+0000","last_became_active":"2023-10-16T08:17:51.754983+0000","last_became_peered":"2023-10-16T08:17:51.754983+0000","last_unstale":"2023-10-16T08:17:52.747874+0000","last_undegraded":"2023-10-16T08:17:52.747874+0000","last_fullsized":"2023-10-16T08:17:52.747874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:10.704012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525629+0000","last_change":"2023-10-16T08:17:51.157530+0000","last_active":"2023-10-16T08:17:53.525629+0000","last_peered":"2023-10-16T08:17:53.525629+0000","last_clean":"2023-10-16T08:17:53.525629+0000","last_became_active":"2023-10-16T08:17:51.157236+0000","last_became_peered":"2023-10-16T08:17:51.157236+0000","last_unstale":"2023-10-16T08:17:53.525629+0000","last_undegraded":"2023-10-16T08:17:53.525629+0000","last_fullsized":"2023-10-16T08:17:53.525629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:51.087792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608911+0000","last_change":"2023-10-16T08:17:51.770479+0000","last_active":"2023-10-16T08:17:53.608911+0000","last_peered":"2023-10-16T08:17:53.608911+0000","last_clean":"2023-10-16T08:17:53.608911+0000","last_became_active":"2023-10-16T08:17:51.769622+0000","last_became_peered":"2023-10-16T08:17:51.769622+0000","last_unstale":"2023-10-16T08:17:53.608911+0000","last_undegraded":"2023-10-16T08:17:53.608911+0000","last_fullsized":"2023-10-16T08:17:53.608911+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:50.948998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748037+0000","last_change":"2023-10-16T08:17:51.166015+0000","last_active":"2023-10-16T08:17:52.748037+0000","last_peered":"2023-10-16T08:17:52.748037+0000","last_clean":"2023-10-16T08:17:52.748037+0000","last_became_active":"2023-10-16T08:17:51.165599+0000","last_became_peered":"2023-10-16T08:17:51.165599+0000","last_unstale":"2023-10-16T08:17:52.748037+0000","last_undegraded":"2023-10-16T08:17:52.748037+0000","last_fullsized":"2023-10-16T08:17:52.748037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:05.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748753+0000","last_change":"2023-10-16T08:17:51.760600+0000","last_active":"2023-10-16T08:17:52.748753+0000","last_peered":"2023-10-16T08:17:52.748753+0000","last_clean":"2023-10-16T08:17:52.748753+0000","last_became_active":"2023-10-16T08:17:51.760353+0000","last_became_peered":"2023-10-16T08:17:51.760353+0000","last_unstale":"2023-10-16T08:17:52.748753+0000","last_undegraded":"2023-10-16T08:17:52.748753+0000","last_fullsized":"2023-10-16T08:17:52.748753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:58:34.233023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608467+0000","last_change":"2023-10-16T08:17:51.163541+0000","last_active":"2023-10-16T08:17:53.608467+0000","last_peered":"2023-10-16T08:17:53.608467+0000","last_clean":"2023-10-16T08:17:53.608467+0000","last_became_active":"2023-10-16T08:17:51.163144+0000","last_became_peered":"2023-10-16T08:17:51.163144+0000","last_unstale":"2023-10-16T08:17:53.608467+0000","last_undegraded":"2023-10-16T08:17:53.608467+0000","last_fullsized":"2023-10-16T08:17:53.608467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:48.829825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525511+0000","last_change":"2023-10-16T08:17:51.746836+0000","last_active":"2023-10-16T08:17:53.525511+0000","last_peered":"2023-10-16T08:17:53.525511+0000","last_clean":"2023-10-16T08:17:53.525511+0000","last_became_active":"2023-10-16T08:17:51.746096+0000","last_became_peered":"2023-10-16T08:17:51.746096+0000","last_unstale":"2023-10-16T08:17:53.525511+0000","last_undegraded":"2023-10-16T08:17:53.525511+0000","last_fullsized":"2023-10-16T08:17:53.525511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:52.230821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609396+0000","last_change":"2023-10-16T08:17:51.166500+0000","last_active":"2023-10-16T08:17:53.609396+0000","last_peered":"2023-10-16T08:17:53.609396+0000","last_clean":"2023-10-16T08:17:53.609396+0000","last_became_active":"2023-10-16T08:17:51.166141+0000","last_became_peered":"2023-10-16T08:17:51.166141+0000","last_unstale":"2023-10-16T08:17:53.609396+0000","last_undegraded":"2023-10-16T08:17:53.609396+0000","last_fullsized":"2023-10-16T08:17:53.609396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:51.223940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608843+0000","last_change":"2023-10-16T08:17:51.754413+0000","last_active":"2023-10-16T08:17:53.608843+0000","last_peered":"2023-10-16T08:17:53.608843+0000","last_clean":"2023-10-16T08:17:53.608843+0000","last_became_active":"2023-10-16T08:17:51.754054+0000","last_became_peered":"2023-10-16T08:17:51.754054+0000","last_unstale":"2023-10-16T08:17:53.608843+0000","last_undegraded":"2023-10-16T08:17:53.608843+0000","last_fullsized":"2023-10-16T08:17:53.608843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:57.348350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607988+0000","last_change":"2023-10-16T08:17:51.168880+0000","last_active":"2023-10-16T08:17:53.607988+0000","last_peered":"2023-10-16T08:17:53.607988+0000","last_clean":"2023-10-16T08:17:53.607988+0000","last_became_active":"2023-10-16T08:17:51.168449+0000","last_became_peered":"2023-10-16T08:17:51.168449+0000","last_unstale":"2023-10-16T08:17:53.607988+0000","last_undegraded":"2023-10-16T08:17:53.607988+0000","last_fullsized":"2023-10-16T08:17:53.607988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:34.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608218+0000","last_change":"2023-10-16T08:17:51.772018+0000","last_active":"2023-10-16T08:17:53.608218+0000","last_peered":"2023-10-16T08:17:53.608218+0000","last_clean":"2023-10-16T08:17:53.608218+0000","last_became_active":"2023-10-16T08:17:51.771721+0000","last_became_peered":"2023-10-16T08:17:51.771721+0000","last_unstale":"2023-10-16T08:17:53.608218+0000","last_undegraded":"2023-10-16T08:17:53.608218+0000","last_fullsized":"2023-10-16T08:17:53.608218+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:16.234591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747586+0000","last_change":"2023-10-16T08:17:51.156660+0000","last_active":"2023-10-16T08:17:52.747586+0000","last_peered":"2023-10-16T08:17:52.747586+0000","last_clean":"2023-10-16T08:17:52.747586+0000","last_became_active":"2023-10-16T08:17:51.156227+0000","last_became_peered":"2023-10-16T08:17:51.156227+0000","last_unstale":"2023-10-16T08:17:52.747586+0000","last_undegraded":"2023-10-16T08:17:52.747586+0000","last_fullsized":"2023-10-16T08:17:52.747586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:48.829825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748485+0000","last_change":"2023-10-16T08:17:51.764715+0000","last_active":"2023-10-16T08:17:52.748485+0000","last_peered":"2023-10-16T08:17:52.748485+0000","last_clean":"2023-10-16T08:17:52.748485+0000","last_became_active":"2023-10-16T08:17:51.764432+0000","last_became_peered":"2023-10-16T08:17:51.764432+0000","last_unstale":"2023-10-16T08:17:52.748485+0000","last_undegraded":"2023-10-16T08:17:52.748485+0000","last_fullsized":"2023-10-16T08:17:52.748485+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:38.450749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607388+0000","last_change":"2023-10-16T08:17:51.163200+0000","last_active":"2023-10-16T08:17:53.607388+0000","last_peered":"2023-10-16T08:17:53.607388+0000","last_clean":"2023-10-16T08:17:53.607388+0000","last_became_active":"2023-10-16T08:17:51.162800+0000","last_became_peered":"2023-10-16T08:17:51.162800+0000","last_unstale":"2023-10-16T08:17:53.607388+0000","last_undegraded":"2023-10-16T08:17:53.607388+0000","last_fullsized":"2023-10-16T08:17:53.607388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:37.857697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605975+0000","last_change":"2023-10-16T08:17:51.770675+0000","last_active":"2023-10-16T08:17:53.605975+0000","last_peered":"2023-10-16T08:17:53.605975+0000","last_clean":"2023-10-16T08:17:53.605975+0000","last_became_active":"2023-10-16T08:17:51.770356+0000","last_became_peered":"2023-10-16T08:17:51.770356+0000","last_unstale":"2023-10-16T08:17:53.605975+0000","last_undegraded":"2023-10-16T08:17:53.605975+0000","last_fullsized":"2023-10-16T08:17:53.605975+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:32.266812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748800+0000","last_change":"2023-10-16T08:17:51.158796+0000","last_active":"2023-10-16T08:17:52.748800+0000","last_peered":"2023-10-16T08:17:52.748800+0000","last_clean":"2023-10-16T08:17:52.748800+0000","last_became_active":"2023-10-16T08:17:51.158083+0000","last_became_peered":"2023-10-16T08:17:51.158083+0000","last_unstale":"2023-10-16T08:17:52.748800+0000","last_undegraded":"2023-10-16T08:17:52.748800+0000","last_fullsized":"2023-10-16T08:17:52.748800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:32.750070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608674+0000","last_change":"2023-10-16T08:17:51.759165+0000","last_active":"2023-10-16T08:17:53.608674+0000","last_peered":"2023-10-16T08:17:53.608674+0000","last_clean":"2023-10-16T08:17:53.608674+0000","last_became_active":"2023-10-16T08:17:51.758877+0000","last_became_peered":"2023-10-16T08:17:51.758877+0000","last_unstale":"2023-10-16T08:17:53.608674+0000","last_undegraded":"2023-10-16T08:17:53.608674+0000","last_fullsized":"2023-10-16T08:17:53.608674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:08.178740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608472+0000","last_change":"2023-10-16T08:17:51.160851+0000","last_active":"2023-10-16T08:17:53.608472+0000","last_peered":"2023-10-16T08:17:53.608472+0000","last_clean":"2023-10-16T08:17:53.608472+0000","last_became_active":"2023-10-16T08:17:51.160381+0000","last_became_peered":"2023-10-16T08:17:51.160381+0000","last_unstale":"2023-10-16T08:17:53.608472+0000","last_undegraded":"2023-10-16T08:17:53.608472+0000","last_fullsized":"2023-10-16T08:17:53.608472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:09.697131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525609+0000","last_change":"2023-10-16T08:17:51.754448+0000","last_active":"2023-10-16T08:17:53.525609+0000","last_peered":"2023-10-16T08:17:53.525609+0000","last_clean":"2023-10-16T08:17:53.525609+0000","last_became_active":"2023-10-16T08:17:51.754079+0000","last_became_peered":"2023-10-16T08:17:51.754079+0000","last_unstale":"2023-10-16T08:17:53.525609+0000","last_undegraded":"2023-10-16T08:17:53.525609+0000","last_fullsized":"2023-10-16T08:17:53.525609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:10.704012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608847+0000","last_change":"2023-10-16T08:17:51.169683+0000","last_active":"2023-10-16T08:17:53.608847+0000","last_peered":"2023-10-16T08:17:53.608847+0000","last_clean":"2023-10-16T08:17:53.608847+0000","last_became_active":"2023-10-16T08:17:51.169362+0000","last_became_peered":"2023-10-16T08:17:51.169362+0000","last_unstale":"2023-10-16T08:17:53.608847+0000","last_undegraded":"2023-10-16T08:17:53.608847+0000","last_fullsized":"2023-10-16T08:17:53.608847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:49.942117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607653+0000","last_change":"2023-10-16T08:17:51.769050+0000","last_active":"2023-10-16T08:17:53.607653+0000","last_peered":"2023-10-16T08:17:53.607653+0000","last_clean":"2023-10-16T08:17:53.607653+0000","last_became_active":"2023-10-16T08:17:51.768735+0000","last_became_peered":"2023-10-16T08:17:51.768735+0000","last_unstale":"2023-10-16T08:17:53.607653+0000","last_undegraded":"2023-10-16T08:17:53.607653+0000","last_fullsized":"2023-10-16T08:17:53.607653+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:42:12.745574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748616+0000","last_change":"2023-10-16T08:17:51.155787+0000","last_active":"2023-10-16T08:17:52.748616+0000","last_peered":"2023-10-16T08:17:52.748616+0000","last_clean":"2023-10-16T08:17:52.748616+0000","last_became_active":"2023-10-16T08:17:51.155517+0000","last_became_peered":"2023-10-16T08:17:51.155517+0000","last_unstale":"2023-10-16T08:17:52.748616+0000","last_undegraded":"2023-10-16T08:17:52.748616+0000","last_fullsized":"2023-10-16T08:17:52.748616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:32.398724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525393+0000","last_change":"2023-10-16T08:17:51.762610+0000","last_active":"2023-10-16T08:17:53.525393+0000","last_peered":"2023-10-16T08:17:53.525393+0000","last_clean":"2023-10-16T08:17:53.525393+0000","last_became_active":"2023-10-16T08:17:51.762359+0000","last_became_peered":"2023-10-16T08:17:51.762359+0000","last_unstale":"2023-10-16T08:17:53.525393+0000","last_undegraded":"2023-10-16T08:17:53.525393+0000","last_fullsized":"2023-10-16T08:17:53.525393+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:06.072317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748629+0000","last_change":"2023-10-16T08:17:51.147628+0000","last_active":"2023-10-16T08:17:52.748629+0000","last_peered":"2023-10-16T08:17:52.748629+0000","last_clean":"2023-10-16T08:17:52.748629+0000","last_became_active":"2023-10-16T08:17:51.147364+0000","last_became_peered":"2023-10-16T08:17:51.147364+0000","last_unstale":"2023-10-16T08:17:52.748629+0000","last_undegraded":"2023-10-16T08:17:52.748629+0000","last_fullsized":"2023-10-16T08:17:52.748629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:55.966020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748413+0000","last_change":"2023-10-16T08:17:51.755564+0000","last_active":"2023-10-16T08:17:52.748413+0000","last_peered":"2023-10-16T08:17:52.748413+0000","last_clean":"2023-10-16T08:17:52.748413+0000","last_became_active":"2023-10-16T08:17:51.754643+0000","last_became_peered":"2023-10-16T08:17:51.754643+0000","last_unstale":"2023-10-16T08:17:52.748413+0000","last_undegraded":"2023-10-16T08:17:52.748413+0000","last_fullsized":"2023-10-16T08:17:52.748413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:08.178740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747560+0000","last_change":"2023-10-16T08:17:51.159148+0000","last_active":"2023-10-16T08:17:52.747560+0000","last_peered":"2023-10-16T08:17:52.747560+0000","last_clean":"2023-10-16T08:17:52.747560+0000","last_became_active":"2023-10-16T08:17:51.158847+0000","last_became_peered":"2023-10-16T08:17:51.158847+0000","last_unstale":"2023-10-16T08:17:52.747560+0000","last_undegraded":"2023-10-16T08:17:52.747560+0000","last_fullsized":"2023-10-16T08:17:52.747560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:42:11.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607968+0000","last_change":"2023-10-16T08:17:51.769837+0000","last_active":"2023-10-16T08:17:53.607968+0000","last_peered":"2023-10-16T08:17:53.607968+0000","last_clean":"2023-10-16T08:17:53.607968+0000","last_became_active":"2023-10-16T08:17:51.769570+0000","last_became_peered":"2023-10-16T08:17:51.769570+0000","last_unstale":"2023-10-16T08:17:53.607968+0000","last_undegraded":"2023-10-16T08:17:53.607968+0000","last_fullsized":"2023-10-16T08:17:53.607968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:33.405605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748968+0000","last_change":"2023-10-16T08:17:51.163789+0000","last_active":"2023-10-16T08:17:52.748968+0000","last_peered":"2023-10-16T08:17:52.748968+0000","last_clean":"2023-10-16T08:17:52.748968+0000","last_became_active":"2023-10-16T08:17:51.163577+0000","last_became_peered":"2023-10-16T08:17:51.163577+0000","last_unstale":"2023-10-16T08:17:52.748968+0000","last_undegraded":"2023-10-16T08:17:52.748968+0000","last_fullsized":"2023-10-16T08:17:52.748968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:52:16.834530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748646+0000","last_change":"2023-10-16T08:17:51.754705+0000","last_active":"2023-10-16T08:17:52.748646+0000","last_peered":"2023-10-16T08:17:52.748646+0000","last_clean":"2023-10-16T08:17:52.748646+0000","last_became_active":"2023-10-16T08:17:51.754369+0000","last_became_peered":"2023-10-16T08:17:51.754369+0000","last_unstale":"2023-10-16T08:17:52.748646+0000","last_undegraded":"2023-10-16T08:17:52.748646+0000","last_fullsized":"2023-10-16T08:17:52.748646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:46.903445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607307+0000","last_change":"2023-10-16T08:17:51.168902+0000","last_active":"2023-10-16T08:17:53.607307+0000","last_peered":"2023-10-16T08:17:53.607307+0000","last_clean":"2023-10-16T08:17:53.607307+0000","last_became_active":"2023-10-16T08:17:51.168615+0000","last_became_peered":"2023-10-16T08:17:51.168615+0000","last_unstale":"2023-10-16T08:17:53.607307+0000","last_undegraded":"2023-10-16T08:17:53.607307+0000","last_fullsized":"2023-10-16T08:17:53.607307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:31.259931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748645+0000","last_change":"2023-10-16T08:17:51.760020+0000","last_active":"2023-10-16T08:17:52.748645+0000","last_peered":"2023-10-16T08:17:52.748645+0000","last_clean":"2023-10-16T08:17:52.748645+0000","last_became_active":"2023-10-16T08:17:51.759750+0000","last_became_peered":"2023-10-16T08:17:51.759750+0000","last_unstale":"2023-10-16T08:17:52.748645+0000","last_undegraded":"2023-10-16T08:17:52.748645+0000","last_fullsized":"2023-10-16T08:17:52.748645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:44.565473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609439+0000","last_change":"2023-10-16T08:17:51.168352+0000","last_active":"2023-10-16T08:17:53.609439+0000","last_peered":"2023-10-16T08:17:53.609439+0000","last_clean":"2023-10-16T08:17:53.609439+0000","last_became_active":"2023-10-16T08:17:51.168066+0000","last_became_peered":"2023-10-16T08:17:51.168066+0000","last_unstale":"2023-10-16T08:17:53.609439+0000","last_undegraded":"2023-10-16T08:17:53.609439+0000","last_fullsized":"2023-10-16T08:17:53.609439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:48.504953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608759+0000","last_change":"2023-10-16T08:17:51.758972+0000","last_active":"2023-10-16T08:17:53.608759+0000","last_peered":"2023-10-16T08:17:53.608759+0000","last_clean":"2023-10-16T08:17:53.608759+0000","last_became_active":"2023-10-16T08:17:51.758652+0000","last_became_peered":"2023-10-16T08:17:51.758652+0000","last_unstale":"2023-10-16T08:17:53.608759+0000","last_undegraded":"2023-10-16T08:17:53.608759+0000","last_fullsized":"2023-10-16T08:17:53.608759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:38.450749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609003+0000","last_change":"2023-10-16T08:17:51.163599+0000","last_active":"2023-10-16T08:17:53.609003+0000","last_peered":"2023-10-16T08:17:53.609003+0000","last_clean":"2023-10-16T08:17:53.609003+0000","last_became_active":"2023-10-16T08:17:51.163175+0000","last_became_peered":"2023-10-16T08:17:51.163175+0000","last_unstale":"2023-10-16T08:17:53.609003+0000","last_undegraded":"2023-10-16T08:17:53.609003+0000","last_fullsized":"2023-10-16T08:17:53.609003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:31.259931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608781+0000","last_change":"2023-10-16T08:17:51.771158+0000","last_active":"2023-10-16T08:17:53.608781+0000","last_peered":"2023-10-16T08:17:53.608781+0000","last_clean":"2023-10-16T08:17:53.608781+0000","last_became_active":"2023-10-16T08:17:51.770813+0000","last_became_peered":"2023-10-16T08:17:51.770813+0000","last_unstale":"2023-10-16T08:17:53.608781+0000","last_undegraded":"2023-10-16T08:17:53.608781+0000","last_fullsized":"2023-10-16T08:17:53.608781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:52.230821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.606171+0000","last_change":"2023-10-16T08:17:51.162707+0000","last_active":"2023-10-16T08:17:53.606171+0000","last_peered":"2023-10-16T08:17:53.606171+0000","last_clean":"2023-10-16T08:17:53.606171+0000","last_became_active":"2023-10-16T08:17:51.160604+0000","last_became_peered":"2023-10-16T08:17:51.160604+0000","last_unstale":"2023-10-16T08:17:53.606171+0000","last_undegraded":"2023-10-16T08:17:53.606171+0000","last_fullsized":"2023-10-16T08:17:53.606171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:02.022071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605994+0000","last_change":"2023-10-16T08:17:51.763433+0000","last_active":"2023-10-16T08:17:53.605994+0000","last_peered":"2023-10-16T08:17:53.605994+0000","last_clean":"2023-10-16T08:17:53.605994+0000","last_became_active":"2023-10-16T08:17:51.763163+0000","last_became_peered":"2023-10-16T08:17:51.763163+0000","last_unstale":"2023-10-16T08:17:53.605994+0000","last_undegraded":"2023-10-16T08:17:53.605994+0000","last_fullsized":"2023-10-16T08:17:53.605994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:38.864578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747730+0000","last_change":"2023-10-16T08:17:51.166826+0000","last_active":"2023-10-16T08:17:52.747730+0000","last_peered":"2023-10-16T08:17:52.747730+0000","last_clean":"2023-10-16T08:17:52.747730+0000","last_became_active":"2023-10-16T08:17:51.166493+0000","last_became_peered":"2023-10-16T08:17:51.166493+0000","last_unstale":"2023-10-16T08:17:52.747730+0000","last_undegraded":"2023-10-16T08:17:52.747730+0000","last_fullsized":"2023-10-16T08:17:52.747730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:34.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607476+0000","last_change":"2023-10-16T08:17:51.771336+0000","last_active":"2023-10-16T08:17:53.607476+0000","last_peered":"2023-10-16T08:17:53.607476+0000","last_clean":"2023-10-16T08:17:53.607476+0000","last_became_active":"2023-10-16T08:17:51.770465+0000","last_became_peered":"2023-10-16T08:17:51.770465+0000","last_unstale":"2023-10-16T08:17:53.607476+0000","last_undegraded":"2023-10-16T08:17:53.607476+0000","last_fullsized":"2023-10-16T08:17:53.607476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:22.519364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605796+0000","last_change":"2023-10-16T08:17:51.163226+0000","last_active":"2023-10-16T08:17:53.605796+0000","last_peered":"2023-10-16T08:17:53.605796+0000","last_clean":"2023-10-16T08:17:53.605796+0000","last_became_active":"2023-10-16T08:17:51.162912+0000","last_became_peered":"2023-10-16T08:17:51.162912+0000","last_unstale":"2023-10-16T08:17:53.605796+0000","last_undegraded":"2023-10-16T08:17:53.605796+0000","last_fullsized":"2023-10-16T08:17:53.605796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:39.792343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525391+0000","last_change":"2023-10-16T08:17:51.760730+0000","last_active":"2023-10-16T08:17:53.525391+0000","last_peered":"2023-10-16T08:17:53.525391+0000","last_clean":"2023-10-16T08:17:53.525391+0000","last_became_active":"2023-10-16T08:17:51.760463+0000","last_became_peered":"2023-10-16T08:17:51.760463+0000","last_unstale":"2023-10-16T08:17:53.525391+0000","last_undegraded":"2023-10-16T08:17:53.525391+0000","last_fullsized":"2023-10-16T08:17:53.525391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:35.540947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605775+0000","last_change":"2023-10-16T08:17:51.167300+0000","last_active":"2023-10-16T08:17:53.605775+0000","last_peered":"2023-10-16T08:17:53.605775+0000","last_clean":"2023-10-16T08:17:53.605775+0000","last_became_active":"2023-10-16T08:17:51.166996+0000","last_became_peered":"2023-10-16T08:17:51.166996+0000","last_unstale":"2023-10-16T08:17:53.605775+0000","last_undegraded":"2023-10-16T08:17:53.605775+0000","last_fullsized":"2023-10-16T08:17:53.605775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:22:58.577489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609268+0000","last_change":"2023-10-16T08:17:51.753909+0000","last_active":"2023-10-16T08:17:53.609268+0000","last_peered":"2023-10-16T08:17:53.609268+0000","last_clean":"2023-10-16T08:17:53.609268+0000","last_became_active":"2023-10-16T08:17:51.753637+0000","last_became_peered":"2023-10-16T08:17:51.753637+0000","last_unstale":"2023-10-16T08:17:53.609268+0000","last_undegraded":"2023-10-16T08:17:53.609268+0000","last_fullsized":"2023-10-16T08:17:53.609268+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:08:46.903445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607830+0000","last_change":"2023-10-16T08:17:51.161844+0000","last_active":"2023-10-16T08:17:53.607830+0000","last_peered":"2023-10-16T08:17:53.607830+0000","last_clean":"2023-10-16T08:17:53.607830+0000","last_became_active":"2023-10-16T08:17:51.161544+0000","last_became_peered":"2023-10-16T08:17:51.161544+0000","last_unstale":"2023-10-16T08:17:53.607830+0000","last_undegraded":"2023-10-16T08:17:53.607830+0000","last_fullsized":"2023-10-16T08:17:53.607830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:04.543107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748505+0000","last_change":"2023-10-16T08:17:51.764623+0000","last_active":"2023-10-16T08:17:52.748505+0000","last_peered":"2023-10-16T08:17:52.748505+0000","last_clean":"2023-10-16T08:17:52.748505+0000","last_became_active":"2023-10-16T08:17:51.764426+0000","last_became_peered":"2023-10-16T08:17:51.764426+0000","last_unstale":"2023-10-16T08:17:52.748505+0000","last_undegraded":"2023-10-16T08:17:52.748505+0000","last_fullsized":"2023-10-16T08:17:52.748505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:57.348350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605824+0000","last_change":"2023-10-16T08:17:51.159106+0000","last_active":"2023-10-16T08:17:53.605824+0000","last_peered":"2023-10-16T08:17:53.605824+0000","last_clean":"2023-10-16T08:17:53.605824+0000","last_became_active":"2023-10-16T08:17:51.158803+0000","last_became_peered":"2023-10-16T08:17:51.158803+0000","last_unstale":"2023-10-16T08:17:53.605824+0000","last_undegraded":"2023-10-16T08:17:53.605824+0000","last_fullsized":"2023-10-16T08:17:53.605824+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:33.860746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747750+0000","last_change":"2023-10-16T08:17:51.757820+0000","last_active":"2023-10-16T08:17:52.747750+0000","last_peered":"2023-10-16T08:17:52.747750+0000","last_clean":"2023-10-16T08:17:52.747750+0000","last_became_active":"2023-10-16T08:17:51.757536+0000","last_became_peered":"2023-10-16T08:17:51.757536+0000","last_unstale":"2023-10-16T08:17:52.747750+0000","last_undegraded":"2023-10-16T08:17:52.747750+0000","last_fullsized":"2023-10-16T08:17:52.747750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:50.948998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.609508+0000","last_change":"2023-10-16T08:17:51.167812+0000","last_active":"2023-10-16T08:17:53.609508+0000","last_peered":"2023-10-16T08:17:53.609508+0000","last_clean":"2023-10-16T08:17:53.609508+0000","last_became_active":"2023-10-16T08:17:51.167544+0000","last_became_peered":"2023-10-16T08:17:51.167544+0000","last_unstale":"2023-10-16T08:17:53.609508+0000","last_undegraded":"2023-10-16T08:17:53.609508+0000","last_fullsized":"2023-10-16T08:17:53.609508+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:01:21.512482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608756+0000","last_change":"2023-10-16T08:17:51.756646+0000","last_active":"2023-10-16T08:17:53.608756+0000","last_peered":"2023-10-16T08:17:53.608756+0000","last_clean":"2023-10-16T08:17:53.608756+0000","last_became_active":"2023-10-16T08:17:51.756370+0000","last_became_peered":"2023-10-16T08:17:51.756370+0000","last_unstale":"2023-10-16T08:17:53.608756+0000","last_undegraded":"2023-10-16T08:17:53.608756+0000","last_fullsized":"2023-10-16T08:17:53.608756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:38.450749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.605897+0000","last_change":"2023-10-16T08:17:51.167361+0000","last_active":"2023-10-16T08:17:53.605897+0000","last_peered":"2023-10-16T08:17:53.605897+0000","last_clean":"2023-10-16T08:17:53.605897+0000","last_became_active":"2023-10-16T08:17:51.167086+0000","last_became_peered":"2023-10-16T08:17:51.167086+0000","last_unstale":"2023-10-16T08:17:53.605897+0000","last_undegraded":"2023-10-16T08:17:53.605897+0000","last_fullsized":"2023-10-16T08:17:53.605897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:51.087792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748801+0000","last_change":"2023-10-16T08:17:51.770518+0000","last_active":"2023-10-16T08:17:52.748801+0000","last_peered":"2023-10-16T08:17:52.748801+0000","last_clean":"2023-10-16T08:17:52.748801+0000","last_became_active":"2023-10-16T08:17:51.770247+0000","last_became_peered":"2023-10-16T08:17:51.770247+0000","last_unstale":"2023-10-16T08:17:52.748801+0000","last_undegraded":"2023-10-16T08:17:52.748801+0000","last_fullsized":"2023-10-16T08:17:52.748801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:10.208587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525477+0000","last_change":"2023-10-16T08:17:51.155822+0000","last_active":"2023-10-16T08:17:53.525477+0000","last_peered":"2023-10-16T08:17:53.525477+0000","last_clean":"2023-10-16T08:17:53.525477+0000","last_became_active":"2023-10-16T08:17:51.155433+0000","last_became_peered":"2023-10-16T08:17:51.155433+0000","last_unstale":"2023-10-16T08:17:53.525477+0000","last_undegraded":"2023-10-16T08:17:53.525477+0000","last_fullsized":"2023-10-16T08:17:53.525477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:45.202816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525575+0000","last_change":"2023-10-16T08:17:51.754218+0000","last_active":"2023-10-16T08:17:53.525575+0000","last_peered":"2023-10-16T08:17:53.525575+0000","last_clean":"2023-10-16T08:17:53.525575+0000","last_became_active":"2023-10-16T08:17:51.753947+0000","last_became_peered":"2023-10-16T08:17:51.753947+0000","last_unstale":"2023-10-16T08:17:53.525575+0000","last_undegraded":"2023-10-16T08:17:53.525575+0000","last_fullsized":"2023-10-16T08:17:53.525575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:22:50.948998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.525602+0000","last_change":"2023-10-16T08:17:51.149101+0000","last_active":"2023-10-16T08:17:53.525602+0000","last_peered":"2023-10-16T08:17:53.525602+0000","last_clean":"2023-10-16T08:17:53.525602+0000","last_became_active":"2023-10-16T08:17:51.148619+0000","last_became_peered":"2023-10-16T08:17:51.148619+0000","last_unstale":"2023-10-16T08:17:53.525602+0000","last_undegraded":"2023-10-16T08:17:53.525602+0000","last_fullsized":"2023-10-16T08:17:53.525602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:34:33.860746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.747499+0000","last_change":"2023-10-16T08:17:51.752263+0000","last_active":"2023-10-16T08:17:52.747499+0000","last_peered":"2023-10-16T08:17:52.747499+0000","last_clean":"2023-10-16T08:17:52.747499+0000","last_became_active":"2023-10-16T08:17:51.751928+0000","last_became_peered":"2023-10-16T08:17:51.751928+0000","last_unstale":"2023-10-16T08:17:52.747499+0000","last_undegraded":"2023-10-16T08:17:52.747499+0000","last_fullsized":"2023-10-16T08:17:52.747499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:16.234591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.749180+0000","last_change":"2023-10-16T08:17:51.156434+0000","last_active":"2023-10-16T08:17:52.749180+0000","last_peered":"2023-10-16T08:17:52.749180+0000","last_clean":"2023-10-16T08:17:52.749180+0000","last_became_active":"2023-10-16T08:17:51.156200+0000","last_became_peered":"2023-10-16T08:17:51.156200+0000","last_unstale":"2023-10-16T08:17:52.749180+0000","last_undegraded":"2023-10-16T08:17:52.749180+0000","last_fullsized":"2023-10-16T08:17:52.749180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:39.792343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607834+0000","last_change":"2023-10-16T08:17:51.767879+0000","last_active":"2023-10-16T08:17:53.607834+0000","last_peered":"2023-10-16T08:17:53.607834+0000","last_clean":"2023-10-16T08:17:53.607834+0000","last_became_active":"2023-10-16T08:17:51.767601+0000","last_became_peered":"2023-10-16T08:17:51.767601+0000","last_unstale":"2023-10-16T08:17:53.607834+0000","last_undegraded":"2023-10-16T08:17:53.607834+0000","last_fullsized":"2023-10-16T08:17:53.607834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:06.072317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.607618+0000","last_change":"2023-10-16T08:17:51.163800+0000","last_active":"2023-10-16T08:17:53.607618+0000","last_peered":"2023-10-16T08:17:53.607618+0000","last_clean":"2023-10-16T08:17:53.607618+0000","last_became_active":"2023-10-16T08:17:51.163229+0000","last_became_peered":"2023-10-16T08:17:51.163229+0000","last_unstale":"2023-10-16T08:17:53.607618+0000","last_undegraded":"2023-10-16T08:17:53.607618+0000","last_fullsized":"2023-10-16T08:17:53.607618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:04.543107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608571+0000","last_change":"2023-10-16T08:17:51.755359+0000","last_active":"2023-10-16T08:17:53.608571+0000","last_peered":"2023-10-16T08:17:53.608571+0000","last_clean":"2023-10-16T08:17:53.608571+0000","last_became_active":"2023-10-16T08:17:51.755060+0000","last_became_peered":"2023-10-16T08:17:51.755060+0000","last_unstale":"2023-10-16T08:17:53.608571+0000","last_undegraded":"2023-10-16T08:17:53.608571+0000","last_fullsized":"2023-10-16T08:17:53.608571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:30.638593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:52.748319+0000","last_change":"2023-10-16T08:17:51.155157+0000","last_active":"2023-10-16T08:17:52.748319+0000","last_peered":"2023-10-16T08:17:52.748319+0000","last_clean":"2023-10-16T08:17:52.748319+0000","last_became_active":"2023-10-16T08:17:51.154881+0000","last_became_peered":"2023-10-16T08:17:51.154881+0000","last_unstale":"2023-10-16T08:17:52.748319+0000","last_undegraded":"2023-10-16T08:17:52.748319+0000","last_fullsized":"2023-10-16T08:17:52.748319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:17.067875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608631+0000","last_change":"2023-10-16T08:17:51.758422+0000","last_active":"2023-10-16T08:17:53.608631+0000","last_peered":"2023-10-16T08:17:53.608631+0000","last_clean":"2023-10-16T08:17:53.608631+0000","last_became_active":"2023-10-16T08:17:51.758162+0000","last_became_peered":"2023-10-16T08:17:51.758162+0000","last_unstale":"2023-10-16T08:17:53.608631+0000","last_undegraded":"2023-10-16T08:17:53.608631+0000","last_fullsized":"2023-10-16T08:17:53.608631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:51.131674+0000","last_clean_scrub_stamp":"2023-10-16T08:17:51.131674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:33.756951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-16T08:17:53.608685+0000","last_change":"2023-10-16T08:17:51.164292+0000","last_active":"2023-10-16T08:17:53.608685+0000","last_peered":"2023-10-16T08:17:53.608685+0000","last_clean":"2023-10-16T08:17:53.608685+0000","last_became_active":"2023-10-16T08:17:51.164013+0000","last_became_peered":"2023-10-16T08:17:51.164013+0000","last_unstale":"2023-10-16T08:17:53.608685+0000","last_undegraded":"2023-10-16T08:17:53.608685+0000","last_fullsized":"2023-10-16T08:17:53.608685+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:17:50.124793+0000","last_clean_scrub_stamp":"2023-10-16T08:17:50.124793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:15.227710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-16T08:17:51.745273+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:17:51.745273+0000","last_peered":"2023-10-16T08:17:51.745273+0000","last_clean":"2023-10-16T08:17:51.745273+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:17:51.745273+0000","last_undegraded":"2023-10-16T08:17:51.745273+0000","last_fullsized":"2023-10-16T08:17:51.745273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":779324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.748}]},{"osd":4,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.628}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82899999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66300000000000003}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.76500000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.78600000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.774,"5min":2.774,"15min":2.774},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.79800000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.62},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81799999999999995}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":187878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.628},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56899999999999995}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.41099999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.44500000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":195565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.96099999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64300000000000002}]},{"osd":6,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.93500000000000005}]},{"osd":7,"last update":"Mon Oct 16 08:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.91100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":187344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.2779999999999996,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":4.8550000000000004,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.399,"5min":4.399,"15min":4.399},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.8220000000000001,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.45900000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.495},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.5579999999999998,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.624}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.5709999999999997,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.7519999999999998,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":194009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.73099999999999998}]},{"osd":5,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":777676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.626}]},{"osd":2,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58699999999999997}]},{"osd":3,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84199999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.94499999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":1.0329999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:17:56.667 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T08:17:56.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=testcephfs1' mds allow 2023-10-16T08:17:56.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 Processor -- start 2023-10-16T08:17:56.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 -- start start 2023-10-16T08:17:56.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00af810 0x7f1dd00cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1dd00cd2e0 0x7f1dd00cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd00aff40 con 0x7f1dd015ad50 2023-10-16T08:17:56.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd00cbe40 con 0x7f1dd00cd2e0 2023-10-16T08:17:56.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd851d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00cdc90 con 0x7f1dd00af810 2023-10-16T08:17:56.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd5ab8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00af810 0x7f1dd00cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd62b9640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f1dd015ad50 0x7f1dd00b3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:53226/0 (socket says 172.21.15.32:53226) 2023-10-16T08:17:56.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd62b9640 1 -- 172.21.15.32:0/4257438945 learned_addr learned my addr 172.21.15.32:0/4257438945 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:56.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd6aba640 1 --2- 172.21.15.32:0/4257438945 >> v2:172.21.15.186:3300/0 conn(0x7f1dd00cd2e0 0x7f1dd00cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd52b7640 1 -- 172.21.15.32:0/4257438945 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028900405 0 0) 0x7f1dd00aff40 con 0x7f1dd015ad50 2023-10-16T08:17:56.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd52b7640 1 -- 172.21.15.32:0/4257438945 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dbc003530 con 0x7f1dd015ad50 2023-10-16T08:17:56.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd6aba640 1 -- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00af810 msgr2=0x7f1dd00cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:56.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd6aba640 1 --2- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00af810 0x7f1dd00cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd6aba640 1 -- 172.21.15.32:0/4257438945 >> v1:172.21.15.186:6789/0 conn(0x7f1dd015ad50 legacy=0x7f1dd00b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd6aba640 1 -- 172.21.15.32:0/4257438945 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00ce840 con 0x7f1dd00cd2e0 2023-10-16T08:17:56.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.849+0000 7f1dd5ab8640 1 --2- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00af810 0x7f1dd00cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:56.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd6aba640 1 --2- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00cd2e0 0x7f1dd00cd750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009dc0 tx=0x7f1dc801c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52965aa0d012ec0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd52b7640 1 -- 172.21.15.32:0/4257438945 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80a58e0 con 0x7f1dd00cd2e0 2023-10-16T08:17:56.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd52b7640 1 -- 172.21.15.32:0/4257438945 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc80a5a40 con 0x7f1dd00cd2e0 2023-10-16T08:17:56.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd52b7640 1 -- 172.21.15.32:0/4257438945 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80a5dd0 con 0x7f1dd00cd2e0 2023-10-16T08:17:56.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 -- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00cd2e0 msgr2=0x7f1dd00cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 --2- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00cd2e0 0x7f1dd00cd750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009dc0 tx=0x7f1dc801c930 comp rx=0 tx=0).stop 2023-10-16T08:17:56.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 -- 172.21.15.32:0/4257438945 shutdown_connections 2023-10-16T08:17:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 --2- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00af810 0x7f1dd00cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 --2- 172.21.15.32:0/4257438945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00cd2e0 0x7f1dd00cd750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 -- 172.21.15.32:0/4257438945 >> 172.21.15.32:0/4257438945 conn(0x7f1dd006e590 msgr2=0x7f1dd006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 -- 172.21.15.32:0/4257438945 shutdown_connections 2023-10-16T08:17:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.853+0000 7f1dd851d640 1 -- 172.21.15.32:0/4257438945 wait complete. 2023-10-16T08:17:56.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 Processor -- start 2023-10-16T08:17:56.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 -- start start 2023-10-16T08:17:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00af810 0x7f1dd00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00cd2e0 0x7f1dd00c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 0x7f1dd0154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00c9fa0 con 0x7f1dd015ad50 2023-10-16T08:17:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd014e8b0 con 0x7f1dd00af810 2023-10-16T08:17:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00c9320 con 0x7f1dd00cd2e0 2023-10-16T08:17:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 0x7f1dd0154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 0x7f1dd0154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50800/0 (socket says 172.21.15.32:50800) 2023-10-16T08:17:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 -- 172.21.15.32:0/3688442267 learned_addr learned my addr 172.21.15.32:0/3688442267 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd5ab8640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00cd2e0 0x7f1dd00c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd62b9640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00af810 0x7f1dd00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 -- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00cd2e0 msgr2=0x7f1dd00c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00cd2e0 0x7f1dd00c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 -- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00af810 msgr2=0x7f1dd00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00af810 0x7f1dd00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:56.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 -- 172.21.15.32:0/3688442267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009420 con 0x7f1dd015ad50 2023-10-16T08:17:56.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd6aba640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 0x7f1dd0154d60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1dc801a560 tx=0x7f1dc80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80ad850 con 0x7f1dd015ad50 2023-10-16T08:17:56.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc80ad9b0 con 0x7f1dd015ad50 2023-10-16T08:17:56.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80add60 con 0x7f1dd015ad50 2023-10-16T08:17:56.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8009a70 con 0x7f1dd015ad50 2023-10-16T08:17:56.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.857+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd00c98e0 con 0x7f1dd015ad50 2023-10-16T08:17:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.861+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd00ca0e0 con 0x7f1dd015ad50 2023-10-16T08:17:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.861+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1dc80b4050 con 0x7f1dd015ad50 2023-10-16T08:17:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.861+0000 7f1dc6ffd640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1db40664a0 0x7f1db4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.861+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f1dc811ebc0 con 0x7f1dd015ad50 2023-10-16T08:17:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.861+0000 7f1dd62b9640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1db40664a0 0x7f1db4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.861+0000 7f1dd62b9640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1db40664a0 0x7f1db4068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0011f0 tx=0x7f1dcc007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:56.865+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1dd00ca0e0 con 0x7f1dd015ad50 2023-10-16T08:17:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.077+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]} v 0) v1 -- 0x7f1dd00ca0e0 con 0x7f1dd015ad50 2023-10-16T08:17:57.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.081+0000 7f1dc6ffd640 1 -- 172.21.15.32:0/3688442267 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 195+0+152 (secure 0 0 0) 0x7f1dd00ca0e0 con 0x7f1dd015ad50 2023-10-16T08:17:57.088 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:17:57.088 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:17:57.088 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-10-16T08:17:57.088 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:17:57.089 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=testcephfs1" 2023-10-16T08:17:57.089 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:17:57.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1db40664a0 msgr2=0x7f1db4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1db40664a0 0x7f1db4068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0011f0 tx=0x7f1dcc007d40 comp rx=0 tx=0).stop 2023-10-16T08:17:57.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 msgr2=0x7f1dd0154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 0x7f1dd0154d60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1dc801a560 tx=0x7f1dc80196d0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 shutdown_connections 2023-10-16T08:17:57.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd015ad50 0x7f1dd0154d60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1db40664a0 0x7f1db4068960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1dd00cd2e0 0x7f1dd00c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 --2- 172.21.15.32:0/3688442267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1dd00af810 0x7f1dd00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 >> 172.21.15.32:0/3688442267 conn(0x7f1dd006e590 msgr2=0x7f1dd00b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 shutdown_connections 2023-10-16T08:17:57.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.085+0000 7f1dd851d640 1 -- 172.21.15.32:0/3688442267 wait complete. 2023-10-16T08:17:57.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:17:57.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.245+0000 7fc50bfb8640 1 Processor -- start 2023-10-16T08:17:57.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.245+0000 7fc50bfb8640 1 -- start start 2023-10-16T08:17:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc50bfb8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc5040b2ad0 0x7fc50406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc50bfb8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50406d000 con 0x7fc5040ce310 2023-10-16T08:17:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc50bfb8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50415d4e0 con 0x7fc50415cf90 2023-10-16T08:17:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc50bfb8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc504160580 con 0x7fc5040b2ad0 2023-10-16T08:17:57.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc50a555640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5040ce310 0x7fc5041609d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60808/0 (socket says 172.21.15.32:60808) 2023-10-16T08:17:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc50a555640 1 -- 172.21.15.32:0/814139119 learned_addr learned my addr 172.21.15.32:0/814139119 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc509d54640 1 --2- 172.21.15.32:0/814139119 >> v2:172.21.15.186:3300/0 conn(0x7fc5040b2ad0 0x7fc50406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc508d52640 1 -- 172.21.15.32:0/814139119 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601307636 0 0) 0x7fc50406d000 con 0x7fc5040ce310 2023-10-16T08:17:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc508d52640 1 -- 172.21.15.32:0/814139119 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f0003530 con 0x7fc5040ce310 2023-10-16T08:17:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc509d54640 1 -- 172.21.15.32:0/814139119 >> v1:172.21.15.186:6789/0 conn(0x7fc50415cf90 legacy=0x7fc50406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc508d52640 1 -- 172.21.15.32:0/814139119 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087841598 0 0) 0x7fc50415d4e0 con 0x7fc50415cf90 2023-10-16T08:17:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc509d54640 1 -- 172.21.15.32:0/814139119 >> v1:172.21.15.32:6789/0 conn(0x7fc5040ce310 legacy=0x7fc5041609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.249+0000 7fc509d54640 1 -- 172.21.15.32:0/814139119 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50415d7c0 con 0x7fc5040b2ad0 2023-10-16T08:17:57.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc509d54640 1 --2- 172.21.15.32:0/814139119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50406b4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc5000097f0 tx=0x7fc50001c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0c23aa31c29b26 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:57.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc508d52640 1 -- 172.21.15.32:0/814139119 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000ab600 con 0x7fc5040b2ad0 2023-10-16T08:17:57.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc508d52640 1 -- 172.21.15.32:0/814139119 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5000a7050 con 0x7fc5040b2ad0 2023-10-16T08:17:57.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc508d52640 1 -- 172.21.15.32:0/814139119 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000ab980 con 0x7fc5040b2ad0 2023-10-16T08:17:57.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- 172.21.15.32:0/814139119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 msgr2=0x7fc50406b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/814139119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50406b4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc5000097f0 tx=0x7fc50001c370 comp rx=0 tx=0).stop 2023-10-16T08:17:57.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- 172.21.15.32:0/814139119 shutdown_connections 2023-10-16T08:17:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/814139119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50406b4b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- 172.21.15.32:0/814139119 >> 172.21.15.32:0/814139119 conn(0x7fc5040cd1c0 msgr2=0x7fc5040cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- 172.21.15.32:0/814139119 shutdown_connections 2023-10-16T08:17:57.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- 172.21.15.32:0/814139119 wait complete. 2023-10-16T08:17:57.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 Processor -- start 2023-10-16T08:17:57.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- start start 2023-10-16T08:17:57.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5040ce310 0x7fc50415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 0x7fc50415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc504151ec0 con 0x7fc50415cf90 2023-10-16T08:17:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50415a250 con 0x7fc5040b2ad0 2023-10-16T08:17:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.253+0000 7fc50bfb8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc504151340 con 0x7fc5040ce310 2023-10-16T08:17:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 0x7fc50415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 0x7fc50415f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50816/0 (socket says 172.21.15.32:50816) 2023-10-16T08:17:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 -- 172.21.15.32:0/3843874537 learned_addr learned my addr 172.21.15.32:0/3843874537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:57.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 -- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5040ce310 msgr2=0x7fc50415f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc509d54640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc509553640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5040ce310 0x7fc50415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5040ce310 0x7fc50415f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 -- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 msgr2=0x7fc50414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 -- 172.21.15.32:0/3843874537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500009420 con 0x7fc50415cf90 2023-10-16T08:17:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc509d54640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc509553640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5040ce310 0x7fc50415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50a555640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 0x7fc50415f7f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc002030 tx=0x7fc4fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:57.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc012400 con 0x7fc50415cf90 2023-10-16T08:17:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc012560 con 0x7fc50415cf90 2023-10-16T08:17:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc504151620 con 0x7fc50415cf90 2023-10-16T08:17:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc012930 con 0x7fc50415cf90 2023-10-16T08:17:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc504151be0 con 0x7fc50415cf90 2023-10-16T08:17:57.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc504074010 con 0x7fc50415cf90 2023-10-16T08:17:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.257+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4fc012a90 con 0x7fc50415cf90 2023-10-16T08:17:57.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.261+0000 7fc4fa7fc640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4e80664a0 0x7fc4e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.261+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fc4fc0847c0 con 0x7fc50415cf90 2023-10-16T08:17:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.261+0000 7fc509d54640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4e80664a0 0x7fc4e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:57.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.261+0000 7fc509d54640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4e80664a0 0x7fc4e8068960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5000ad000 tx=0x7fc500008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:57.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.265+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc4fc051b70 con 0x7fc50415cf90 2023-10-16T08:17:57.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.453+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5040c4300 con 0x7fc50415cf90 2023-10-16T08:17:57.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.453+0000 7fc4fa7fc640 1 -- 172.21.15.32:0/3843874537 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fc4fc055190 con 0x7fc50415cf90 2023-10-16T08:17:57.460 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:17:57.460 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:17:54.554739+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:17:57.461 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2023-10-16T08:17:57.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4e80664a0 msgr2=0x7fc4e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4e80664a0 0x7fc4e8068960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5000ad000 tx=0x7fc500008040 comp rx=0 tx=0).stop 2023-10-16T08:17:57.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 msgr2=0x7fc50415f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 0x7fc50415f7f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc002030 tx=0x7fc4fc00a430 comp rx=0 tx=0).stop 2023-10-16T08:17:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 shutdown_connections 2023-10-16T08:17:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc50415cf90 0x7fc50415f7f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc4e80664a0 0x7fc4e8068960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5040ce310 0x7fc50415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 --2- 172.21.15.32:0/3843874537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5040b2ad0 0x7fc50414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 >> 172.21.15.32:0/3843874537 conn(0x7fc5040cd1c0 msgr2=0x7fc5041527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 shutdown_connections 2023-10-16T08:17:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:57.457+0000 7fc50bfb8640 1 -- 172.21.15.32:0/3843874537 wait complete. 2023-10-16T08:17:57.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T08:17:51.541347+0000', 'modified': '2023-10-16T08:17:54.554739+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5520': {'gid': 5520, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/1556343693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 1556343693}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 1556343693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'testcephfs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T08:17:57.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T08:17:57.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T08:17:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.021+0000 7efe28bb8640 1 Processor -- start 2023-10-16T08:17:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe28bb8640 1 -- start start 2023-10-16T08:17:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe28bb8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe241591e0 0x7efe240c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe28bb8640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efe240ca430 0x7efe24071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe28bb8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe24071020 con 0x7efe240721b0 2023-10-16T08:17:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe28bb8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe240c8e50 con 0x7efe241591e0 2023-10-16T08:17:58.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe28bb8640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe240c9130 con 0x7efe240ca430 2023-10-16T08:17:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe241591e0 0x7efe240c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe241591e0 0x7efe240c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50828/0 (socket says 172.21.15.32:50828) 2023-10-16T08:17:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 -- 172.21.15.32:0/3749521578 learned_addr learned my addr 172.21.15.32:0/3749521578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe1bfff640 1 --2- 172.21.15.32:0/3749521578 >> v2:172.21.15.186:3300/0 conn(0x7efe240ca430 0x7efe24071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 -- 172.21.15.32:0/3749521578 >> v2:172.21.15.186:3300/0 conn(0x7efe240ca430 msgr2=0x7efe24071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 --2- 172.21.15.32:0/3749521578 >> v2:172.21.15.186:3300/0 conn(0x7efe240ca430 0x7efe24071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 -- 172.21.15.32:0/3749521578 >> v1:172.21.15.186:6789/0 conn(0x7efe240721b0 legacy=0x7efe240b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.025+0000 7efe2259c640 1 -- 172.21.15.32:0/3749521578 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe240c9410 con 0x7efe241591e0 2023-10-16T08:17:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe2259c640 1 --2- 172.21.15.32:0/3749521578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe240c9ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efe2415dc70 tx=0x7efe1401d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdc4e6233cc863c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe21d9b640 1 -- 172.21.15.32:0/3749521578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140a6830 con 0x7efe241591e0 2023-10-16T08:17:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe21d9b640 1 -- 172.21.15.32:0/3749521578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe140a6990 con 0x7efe241591e0 2023-10-16T08:17:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe21d9b640 1 -- 172.21.15.32:0/3749521578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140a6d20 con 0x7efe241591e0 2023-10-16T08:17:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 -- 172.21.15.32:0/3749521578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 msgr2=0x7efe240c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3749521578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe240c9ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efe2415dc70 tx=0x7efe1401d890 comp rx=0 tx=0).stop 2023-10-16T08:17:58.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 -- 172.21.15.32:0/3749521578 shutdown_connections 2023-10-16T08:17:58.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3749521578 >> v2:172.21.15.186:3300/0 conn(0x7efe240ca430 0x7efe24071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3749521578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe240c9ef0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 -- 172.21.15.32:0/3749521578 >> 172.21.15.32:0/3749521578 conn(0x7efe2406c190 msgr2=0x7efe2406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 -- 172.21.15.32:0/3749521578 shutdown_connections 2023-10-16T08:17:58.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 -- 172.21.15.32:0/3749521578 wait complete. 2023-10-16T08:17:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.029+0000 7efe28bb8640 1 Processor -- start 2023-10-16T08:17:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 -- start start 2023-10-16T08:17:58.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe240721b0 0x7efe2414aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe240ca430 0x7efe241518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe241f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe24167ea0 con 0x7efe241591e0 2023-10-16T08:17:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241f63c0 con 0x7efe240721b0 2023-10-16T08:17:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe24167640 con 0x7efe240ca430 2023-10-16T08:17:58.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe241f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe241f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50832/0 (socket says 172.21.15.32:50832) 2023-10-16T08:17:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 -- 172.21.15.32:0/3624607892 learned_addr learned my addr 172.21.15.32:0/3624607892 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe2259c640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe240721b0 0x7efe2414aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe1bfff640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe240ca430 0x7efe241518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 -- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe240ca430 msgr2=0x7efe241518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe240ca430 0x7efe241518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 -- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe240721b0 msgr2=0x7efe2414aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe240721b0 0x7efe2414aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 -- 172.21.15.32:0/3624607892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1400a670 con 0x7efe241591e0 2023-10-16T08:17:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe22d9d640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe241f5e80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c780 tx=0x7efe1c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe1affd640 1 -- 172.21.15.32:0/3624607892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c004b20 con 0x7efe241591e0 2023-10-16T08:17:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe1affd640 1 -- 172.21.15.32:0/3624607892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c004c80 con 0x7efe241591e0 2023-10-16T08:17:58.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe1affd640 1 -- 172.21.15.32:0/3624607892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00aa90 con 0x7efe241591e0 2023-10-16T08:17:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe241678e0 con 0x7efe241591e0 2023-10-16T08:17:58.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.033+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe240b3850 con 0x7efe241591e0 2023-10-16T08:17:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe1affd640 1 -- 172.21.15.32:0/3624607892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efe1c00abf0 con 0x7efe241591e0 2023-10-16T08:17:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe1affd640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe040664a0 0x7efe04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe1affd640 1 -- 172.21.15.32:0/3624607892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7efe1c083fa0 con 0x7efe241591e0 2023-10-16T08:17:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe2bf45000 1 Processor -- start 2023-10-16T08:17:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe2259c640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe040664a0 0x7efe04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe2bf45000 1 -- start start 2023-10-16T08:17:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.037+0000 7efe2259c640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe040664a0 0x7efe04068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe140b0000 tx=0x7efe14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba56c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba0c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba0d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b801cc0 con 0x55862ba0cc50 2023-10-16T08:17:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b864920 con 0x55862ba568a0 2023-10-16T08:17:58.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55862b861e50 con 0x55862ba571c0 2023-10-16T08:17:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba0d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba0d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50836/0 (socket says 172.21.15.32:50836) 2023-10-16T08:17:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 -- 172.21.15.32:0/3380729658 learned_addr learned my addr 172.21.15.32:0/3380729658 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2259c640 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba56c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe1bfff640 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba0c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 -- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 msgr2=0x55862ba0c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba0c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 -- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 msgr2=0x55862ba56c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba56c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 -- 172.21.15.32:0/3380729658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862b556890 con 0x55862ba0cc50 2023-10-16T08:17:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe22d9d640 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba0d030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efe1c007fd0 tx=0x7efe1c00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adcc9babf0d5e43 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe03fff640 1 -- 172.21.15.32:0/3380729658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0820c0 con 0x55862ba0cc50 2023-10-16T08:17:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe03fff640 1 -- 172.21.15.32:0/3380729658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c05d270 con 0x55862ba0cc50 2023-10-16T08:17:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe03fff640 1 -- 172.21.15.32:0/3380729658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0533c0 con 0x55862ba0cc50 2023-10-16T08:17:58.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- 172.21.15.32:0/3380729658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 msgr2=0x55862ba0d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba0d030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efe1c007fd0 tx=0x7efe1c00cef0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- 172.21.15.32:0/3380729658 shutdown_connections 2023-10-16T08:17:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba0d030 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba0c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3380729658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba56c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- 172.21.15.32:0/3380729658 >> 172.21.15.32:0/3380729658 conn(0x55862ba03fc0 msgr2=0x55862ba0aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- 172.21.15.32:0/3380729658 shutdown_connections 2023-10-16T08:17:58.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- 172.21.15.32:0/3380729658 wait complete. 2023-10-16T08:17:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 -- start start 2023-10-16T08:17:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.041+0000 7efe2bf45000 1 Processor -- start 2023-10-16T08:17:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba0cc50 0x55862ba11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba12460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 0x55862ba44d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b85d740 con 0x55862ba571c0 2023-10-16T08:17:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b81cd10 con 0x55862ba568a0 2023-10-16T08:17:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55862b81c9b0 con 0x55862ba0cc50 2023-10-16T08:17:58.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 10 client.0 _ll_register_callbacks cb 0x55862b9f8390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:17:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 0x55862ba44d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 0x55862ba44d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50840/0 (socket says 172.21.15.32:50840) 2023-10-16T08:17:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 -- 172.21.15.32:0/854074306 learned_addr learned my addr 172.21.15.32:0/854074306 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2259c640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba0cc50 0x55862ba11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe1bfff640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba12460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 -- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba0cc50 msgr2=0x55862ba11f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba0cc50 0x55862ba11f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 -- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 msgr2=0x55862ba12460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba12460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 -- 172.21.15.32:0/854074306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe240b3850 con 0x55862ba571c0 2023-10-16T08:17:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2259c640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba0cc50 0x55862ba11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe1bfff640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba12460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe22d9d640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 0x55862ba44d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efe1c054fb0 tx=0x7efe1c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28e01da68026f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:17:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 1 -- 172.21.15.32:0/854074306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0533c0 con 0x55862ba571c0 2023-10-16T08:17:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 1 -- 172.21.15.32:0/854074306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7efe1c05b0b0 con 0x55862ba571c0 2023-10-16T08:17:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55862b811370 con 0x55862ba571c0 2023-10-16T08:17:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 1 -- 172.21.15.32:0/854074306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1c07c080 con 0x55862ba571c0 2023-10-16T08:17:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 10 client.5682 fetch_fsmap learned FSMap version 18 2023-10-16T08:17:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55862b81f460 con 0x55862ba571c0 2023-10-16T08:17:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 1 -- 172.21.15.32:0/854074306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c061020 con 0x55862ba571c0 2023-10-16T08:17:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 1 -- 172.21.15.32:0/854074306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 18) v1 ==== 3299+0+0 (secure 0 0 0) 0x7efe1c019070 con 0x55862ba571c0 2023-10-16T08:17:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 10 client.5682 fetch_fsmap finished waiting for FSMap version 18 2023-10-16T08:17:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 10 client.5682 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T08:17:58.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 20 client.5682 populate_metadata read hostname 'smithi032' 2023-10-16T08:17:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba66940 0x55862ba68d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 4 client.5682 mds_command: new command op to 5520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T08:17:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 --> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55862b7f72d0 con 0x55862ba66940 2023-10-16T08:17:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2259c640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba66940 0x55862ba68d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2259c640 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba66940 0x55862ba68d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6654ee0f0a9ae81 server_cookie=8a521b5f7fc02188 in_seq=0 out_seq=0 2023-10-16T08:17:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 10 client.5682 ms_handle_connect on v2:172.21.15.32:6834/1556343693 2023-10-16T08:17:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 1 -- 172.21.15.32:0/854074306 <== mds.0 v2:172.21.15.32:6834/1556343693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55862b7f72d0 con 0x55862ba66940 2023-10-16T08:17:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe017fa640 10 client.5682 handle_command_reply: tid=0 2023-10-16T08:17:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.045+0000 7efe2bf45000 1 client.5682 shutdown 2023-10-16T08:17:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 msgr2=0x55862ba44d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 0x55862ba44d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efe1c054fb0 tx=0x7efe1c007f60 comp rx=0 tx=0).stop 2023-10-16T08:17:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 shutdown_connections 2023-10-16T08:17:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba66940 0x55862ba68d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba571c0 0x55862ba44d80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba12460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 --2- 172.21.15.32:0/854074306 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba0cc50 0x55862ba11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 >> 172.21.15.32:0/854074306 conn(0x55862ba03fc0 msgr2=0x55862ba0b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe017fa640 0 client.5682 ms_handle_reset on v2:172.21.15.32:6834/1556343693 2023-10-16T08:17:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 shutdown_connections 2023-10-16T08:17:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 1 -- 172.21.15.32:0/854074306 wait complete. 2023-10-16T08:17:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.049+0000 7efe2bf45000 20 client.5682 trim_cache size 0 max 16384 2023-10-16T08:17:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.061+0000 7efe2bf45000 1 Processor -- start 2023-10-16T08:17:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.061+0000 7efe2bf45000 1 -- start start 2023-10-16T08:17:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba69d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba6a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b517940 con 0x55862ba0cc50 2023-10-16T08:17:58.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b8602a0 con 0x55862ba568a0 2023-10-16T08:17:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55862ba116e0 con 0x55862ba571c0 2023-10-16T08:17:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba69d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba69d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50848/0 (socket says 172.21.15.32:50848) 2023-10-16T08:17:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 -- 172.21.15.32:0/3452243566 learned_addr learned my addr 172.21.15.32:0/3452243566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 -- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 msgr2=0x55862ba3f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba3f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 -- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 msgr2=0x55862ba6a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe1bfff640 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba6a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba6a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 -- 172.21.15.32:0/3452243566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862b805760 con 0x55862ba0cc50 2023-10-16T08:17:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe1bfff640 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba6a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:17:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2259c640 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba69d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe1400a1c0 tx=0x7efe14004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e337700d0da6b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe18ff9640 1 -- 172.21.15.32:0/3452243566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140c9af0 con 0x55862ba0cc50 2023-10-16T08:17:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe18ff9640 1 -- 172.21.15.32:0/3452243566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe140be040 con 0x55862ba0cc50 2023-10-16T08:17:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe18ff9640 1 -- 172.21.15.32:0/3452243566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140c9c50 con 0x55862ba0cc50 2023-10-16T08:17:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- 172.21.15.32:0/3452243566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 msgr2=0x55862ba69d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba69d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe1400a1c0 tx=0x7efe14004740 comp rx=0 tx=0).stop 2023-10-16T08:17:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- 172.21.15.32:0/3452243566 shutdown_connections 2023-10-16T08:17:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba3f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba568a0 0x55862ba6a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 --2- 172.21.15.32:0/3452243566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba0cc50 0x55862ba69d40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- 172.21.15.32:0/3452243566 >> 172.21.15.32:0/3452243566 conn(0x55862ba03fc0 msgr2=0x55862ba63e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- 172.21.15.32:0/3452243566 shutdown_connections 2023-10-16T08:17:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- 172.21.15.32:0/3452243566 wait complete. 2023-10-16T08:17:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 -- start start 2023-10-16T08:17:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.065+0000 7efe2bf45000 1 Processor -- start 2023-10-16T08:17:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba0cc50 0x55862bab5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 0x55862bab5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ba10f80 con 0x55862ba568a0 2023-10-16T08:17:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862b8628b0 con 0x55862ba0cc50 2023-10-16T08:17:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x55862b861740 con 0x55862ba571c0 2023-10-16T08:17:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 10 client.0 _ll_register_callbacks cb 0x7efe1c051360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:17:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 0x55862bab5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba0cc50 0x55862bab5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 0x55862bab5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50860/0 (socket says 172.21.15.32:50860) 2023-10-16T08:17:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe1bfff640 1 -- 172.21.15.32:0/1966872958 learned_addr learned my addr 172.21.15.32:0/1966872958 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe22d9d640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 -- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 msgr2=0x55862ba426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 -- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 msgr2=0x55862bab5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 0x55862bab5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 -- 172.21.15.32:0/1966872958 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe240c9410 con 0x55862ba0cc50 2023-10-16T08:17:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe1bfff640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 0x55862bab5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:17:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2259c640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba0cc50 0x55862bab5350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe1401b4c0 tx=0x7efe1401c0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26ed05a48a94de1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T08:17:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 1 -- 172.21.15.32:0/1966872958 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140c9ce0 con 0x55862ba0cc50 2023-10-16T08:17:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 1 -- 172.21.15.32:0/1966872958 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7efe140c9e40 con 0x55862ba0cc50 2023-10-16T08:17:58.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55862b7f56f0 con 0x55862ba0cc50 2023-10-16T08:17:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 1 -- 172.21.15.32:0/1966872958 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe140c9160 con 0x55862ba0cc50 2023-10-16T08:17:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 10 client.5611 fetch_fsmap learned FSMap version 18 2023-10-16T08:17:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55862b556cc0 con 0x55862ba0cc50 2023-10-16T08:17:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 1 -- 172.21.15.32:0/1966872958 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55862b7f56f0 con 0x55862ba0cc50 2023-10-16T08:17:58.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 1 -- 172.21.15.32:0/1966872958 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 18) v1 ==== 3299+0+0 (secure 0 0 0) 0x7efe140c3040 con 0x55862ba0cc50 2023-10-16T08:17:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 10 client.5611 fetch_fsmap finished waiting for FSMap version 18 2023-10-16T08:17:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 10 client.5611 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T08:17:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 20 client.5611 populate_metadata read hostname 'smithi032' 2023-10-16T08:17:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba43080 0x55862ba43460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 4 client.5611 mds_command: new command op to 5520 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T08:17:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 --> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55862b856720 con 0x55862ba43080 2023-10-16T08:17:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe1bfff640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba43080 0x55862ba43460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe1bfff640 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba43080 0x55862ba43460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67c962780dc664b server_cookie=42ea058d187e0cfe in_seq=0 out_seq=0 2023-10-16T08:17:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 10 client.5611 ms_handle_connect on v2:172.21.15.32:6834/1556343693 2023-10-16T08:17:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 1 -- 172.21.15.32:0/1966872958 <== mds.0 v2:172.21.15.32:6834/1556343693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+336 (crc 0 0 0) 0x55862b856720 con 0x55862ba43080 2023-10-16T08:17:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe017fa640 10 client.5611 handle_command_reply: tid=0 2023-10-16T08:17:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.069+0000 7efe2bf45000 1 client.5611 shutdown 2023-10-16T08:17:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba0cc50 msgr2=0x55862bab5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba0cc50 0x55862bab5350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe1401b4c0 tx=0x7efe1401c0d0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 shutdown_connections 2023-10-16T08:17:58.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x55862ba571c0 0x55862ba426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:6834/1556343693,v1:172.21.15.32:6835/1556343693] conn(0x55862ba43080 0x55862ba43460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55862ba568a0 0x55862bab5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 --2- 172.21.15.32:0/1966872958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55862ba0cc50 0x55862bab5350 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 >> 172.21.15.32:0/1966872958 conn(0x55862ba03fc0 msgr2=0x55862ba68d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe017fa640 0 client.5611 ms_handle_reset on v2:172.21.15.32:6834/1556343693 2023-10-16T08:17:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 shutdown_connections 2023-10-16T08:17:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 1 -- 172.21.15.32:0/1966872958 wait complete. 2023-10-16T08:17:58.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe2bf45000 20 client.5611 trim_cache size 0 max 16384 2023-10-16T08:17:58.093 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:17:58.093 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "943c836a-5c4c-4cb7-a190-9643b3476f99", 2023-10-16T08:17:58.093 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-10-16T08:17:58.093 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5520, 2023-10-16T08:17:58.094 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-10-16T08:17:58.094 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-10-16T08:17:58.094 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1", 2023-10-16T08:17:58.094 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9192782399999997, 2023-10-16T08:17:58.094 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 18, 2023-10-16T08:17:58.095 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 39, 2023-10-16T08:17:58.095 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 39, 2023-10-16T08:17:58.095 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 13.772950850000001 2023-10-16T08:17:58.095 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:17:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe040664a0 msgr2=0x7efe04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe040664a0 0x7efe04068960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe140b0000 tx=0x7efe14008040 comp rx=0 tx=0).stop 2023-10-16T08:17:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 msgr2=0x7efe241f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe241f5e80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c780 tx=0x7efe1c00cc40 comp rx=0 tx=0).stop 2023-10-16T08:17:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 shutdown_connections 2023-10-16T08:17:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe241591e0 0x7efe241f5e80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efe040664a0 0x7efe04068960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efe240ca430 0x7efe241518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 --2- 172.21.15.32:0/3624607892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe240721b0 0x7efe2414aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 >> 172.21.15.32:0/3624607892 conn(0x7efe2406c190 msgr2=0x7efe240cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 shutdown_connections 2023-10-16T08:17:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.073+0000 7efe28bb8640 1 -- 172.21.15.32:0/3624607892 wait complete. 2023-10-16T08:17:58.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:17:58.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 Processor -- start 2023-10-16T08:17:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 -- start start 2023-10-16T08:17:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f983015ca70 0x7f98300d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300cf7a0 0x7f98300c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98300c65f0 con 0x7f98300d3550 2023-10-16T08:17:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98300ce820 con 0x7f983015ca70 2023-10-16T08:17:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9837980640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300cdf80 con 0x7f98300cf7a0 2023-10-16T08:17:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f983571c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f983015ca70 0x7f98300d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9834f1b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98300d3550 0x7f98300cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60818/0 (socket says 172.21.15.32:60818) 2023-10-16T08:17:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9834f1b640 1 -- 172.21.15.32:0/2020766977 learned_addr learned my addr 172.21.15.32:0/2020766977 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9835f1d640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300cf7a0 0x7f98300c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f983571c640 1 -- 172.21.15.32:0/2020766977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300cf7a0 msgr2=0x7f98300c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f983571c640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300cf7a0 0x7f98300c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f983571c640 1 -- 172.21.15.32:0/2020766977 >> v1:172.21.15.32:6789/0 conn(0x7f98300d3550 legacy=0x7f98300cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f983571c640 1 -- 172.21.15.32:0/2020766977 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300ce260 con 0x7f983015ca70 2023-10-16T08:17:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9827fff640 1 -- 172.21.15.32:0/2020766977 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432699088 0 0) 0x7f98300c65f0 con 0x7f98300d3550 2023-10-16T08:17:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.237+0000 7f9835f1d640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300cf7a0 0x7f98300c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:17:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f983571c640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f983015ca70 0x7f98300d3010 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f982c0097a0 tx=0x7f982c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6735c38258d0eb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9827fff640 1 -- 172.21.15.32:0/2020766977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0ad640 con 0x7f983015ca70 2023-10-16T08:17:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9827fff640 1 -- 172.21.15.32:0/2020766977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c0ad7a0 con 0x7f983015ca70 2023-10-16T08:17:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9827fff640 1 -- 172.21.15.32:0/2020766977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0adb30 con 0x7f983015ca70 2023-10-16T08:17:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 -- 172.21.15.32:0/2020766977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f983015ca70 msgr2=0x7f98300d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f983015ca70 0x7f98300d3010 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f982c0097a0 tx=0x7f982c01c430 comp rx=0 tx=0).stop 2023-10-16T08:17:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 -- 172.21.15.32:0/2020766977 shutdown_connections 2023-10-16T08:17:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300cf7a0 0x7f98300c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 --2- 172.21.15.32:0/2020766977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f983015ca70 0x7f98300d3010 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 -- 172.21.15.32:0/2020766977 >> 172.21.15.32:0/2020766977 conn(0x7f983009bc50 msgr2=0x7f98300d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 -- 172.21.15.32:0/2020766977 shutdown_connections 2023-10-16T08:17:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.241+0000 7f9837980640 1 -- 172.21.15.32:0/2020766977 wait complete. 2023-10-16T08:17:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 Processor -- start 2023-10-16T08:17:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- start start 2023-10-16T08:17:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 0x7f98301560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300d3550 0x7f983014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f983015ca70 0x7f983014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300cc850 con 0x7f98300cf7a0 2023-10-16T08:17:58.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300c99d0 con 0x7f983015ca70 2023-10-16T08:17:58.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300c9c20 con 0x7f98300d3550 2023-10-16T08:17:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 0x7f98301560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 0x7f98301560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50870/0 (socket says 172.21.15.32:50870) 2023-10-16T08:17:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 -- 172.21.15.32:0/2536425315 learned_addr learned my addr 172.21.15.32:0/2536425315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9835f1d640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f983015ca70 0x7f983014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9834f1b640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300d3550 0x7f983014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 -- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300d3550 msgr2=0x7f983014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300d3550 0x7f983014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 -- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f983015ca70 msgr2=0x7f983014ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f983015ca70 0x7f983014ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 -- 172.21.15.32:0/2536425315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0093b0 con 0x7f98300cf7a0 2023-10-16T08:17:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f983571c640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 0x7f98301560c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f982c01a060 tx=0x7f982c0191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0b6610 con 0x7f98300cf7a0 2023-10-16T08:17:58.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c0a6040 con 0x7f98300cf7a0 2023-10-16T08:17:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0b69b0 con 0x7f98300cf7a0 2023-10-16T08:17:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98300c9f00 con 0x7f98300cf7a0 2023-10-16T08:17:58.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98300ca4c0 con 0x7f98300cf7a0 2023-10-16T08:17:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.245+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98300ca6b0 con 0x7f98300cf7a0 2023-10-16T08:17:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.249+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f982c0b4040 con 0x7f98300cf7a0 2023-10-16T08:17:58.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.249+0000 7f9825ffb640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98140664a0 0x7f9814068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.249+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f982c11e3c0 con 0x7f98300cf7a0 2023-10-16T08:17:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.249+0000 7f9834f1b640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98140664a0 0x7f9814068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:58.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.249+0000 7f9834f1b640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98140664a0 0x7f9814068960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f983015b780 tx=0x7f9820007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.253+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f982c0eb810 con 0x7f98300cf7a0 2023-10-16T08:17:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.445+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9830074010 con 0x7f98300cf7a0 2023-10-16T08:17:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.445+0000 7f9825ffb640 1 -- 172.21.15.32:0/2536425315 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+88 (secure 0 0 0) 0x7f982c0b2070 con 0x7f98300cf7a0 2023-10-16T08:17:58.453 INFO:teuthology.orchestra.run.smithi032.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T08:17:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98140664a0 msgr2=0x7f9814068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98140664a0 0x7f9814068960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f983015b780 tx=0x7f9820007d60 comp rx=0 tx=0).stop 2023-10-16T08:17:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 msgr2=0x7f98301560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 0x7f98301560c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f982c01a060 tx=0x7f982c0191d0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 shutdown_connections 2023-10-16T08:17:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f983015ca70 0x7f983014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98140664a0 0x7f9814068960 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98300d3550 0x7f983014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 --2- 172.21.15.32:0/2536425315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98300cf7a0 0x7f98301560c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 >> 172.21.15.32:0/2536425315 conn(0x7f983009bc50 msgr2=0x7f98300d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 shutdown_connections 2023-10-16T08:17:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:58.449+0000 7f9837980640 1 -- 172.21.15.32:0/2536425315 wait complete. 2023-10-16T08:17:58.473 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:17:58.474 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:17:58.474 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:17:58.474 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:17:58.475 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:58.475 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:17:58.475 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:17:58.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:17:58.476 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:17:58.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:58.477 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:17:58.488 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:17:58.488 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:17:58.488 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:17:58.488 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.488 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:17:58.489 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.489 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:58.489 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:58.489 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:17:58.489 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:58.490 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:58.490 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:17:58.490 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:17:58.490 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:58.491 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:17:58.491 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:17:58.491 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42327sec preferred_lft 42327sec 2023-10-16T08:17:58.491 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:17:58.491 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.492 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:17:58.492 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:58.492 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:17:58.492 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:17:58.492 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:58.493 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:17:58.493 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.493 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:17:58.493 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:17:58.494 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:17:58.494 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.494 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:17:58.494 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.494 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:17:58.495 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:58.495 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:17:58.495 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.495 INFO:teuthology.orchestra.run.smithi032.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:17:58.495 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:58.496 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T08:17:58.496 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:17:58.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:58.497 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:17:58.548 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:17:58.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:58.551 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:17:58.603 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:17:58.604 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:17:58.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:58.605 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:17:58.605 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:17:58.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:17:58.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:17:58.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:17:58.606 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:17:58.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:58.702 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:17:58.712 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:17:58.712 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:17:58.712 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:17:58.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:58.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:17:58.775 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:17:58.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:58.776 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:58.826 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:17:58.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:17:58.826 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:58.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:17:58.947 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:58.999 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:17:58.999 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:17:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:17:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:17:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:17:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:17:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:17:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:17:59.001 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:17:59.001 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:17:59.001 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:17:59.001 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:17:59.001 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:59.002 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:17:59.002 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:17:59.002 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:17:59.002 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:17:59.002 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:17:59.003 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:59.003 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:17:59.003 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:59.003 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:17:59.003 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:17:59.004 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:17:59.004 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:17:59.004 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:17:59.004 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:17:59.004 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:17:59.005 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:17:59.005 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:59.005 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:59.005 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:59.005 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:59.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:59.007 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:17:59.062 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:17:59.062 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:17:59.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:17:59.137 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:17:59.187 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:17:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:17:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:17:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:17:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:17:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:17:59.189 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:17:59.189 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:17:59.189 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:17:59.189 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:17:59.189 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:17:59.190 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:17:59.190 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:17:59.190 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:17:59.190 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:17:59.190 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:17:59.191 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:17:59.191 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:17:59.191 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:17:59.191 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:17:59.191 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:17:59.192 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:17:59.192 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:17:59.192 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:17:59.192 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:17:59.192 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:17:59.193 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:17:59.193 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:17:59.193 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:17:59.193 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:59.193 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:17:59.194 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:59.194 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:17:59.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:59.196 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:17:59.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:17:59.205+0000 7fbdf742d580 -1 init, newargv = 0x5617b8112aa0 newargc=15 2023-10-16T08:17:59.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48255]: starting ceph client 2023-10-16T08:17:59.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48255]: starting fuse 2023-10-16T08:17:59.222 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:17:59.223 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:17:59.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:17:59.224 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:17:59.224 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:17:59.224 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:17:59.225 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:17:59.225 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:59.226 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:17:59.226 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:17:59.227 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:17:59.227 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:17:59.227 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:59.228 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:17:59.228 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:17:59.228 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:17:59.229 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:17:59.229 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:59.229 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:17:59.230 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:17:59.230 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:17:59.231 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:17:59.231 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:17:59.231 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:17:59.232 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:17:59.232 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:17:59.232 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:17:59.233 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:17:59.233 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:17:59.349 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.48255.asok 2023-10-16T08:17:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:17:59.360 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48255.asok 2023-10-16T08:17:59.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:59.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48255.asok status 2023-10-16T08:17:59.538 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:17:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:17:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:17:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:17:59.539 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:17:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:17:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:17:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "48255", 2023-10-16T08:17:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:17:59.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:17:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:17:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5706, 2023-10-16T08:17:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:17:59.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:17:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:17:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 5706 2023-10-16T08:17:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:17:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:17:59.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:17:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 77630305 2023-10-16T08:17:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:17:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:17:59.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:17:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:17:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 77630305 2023-10-16T08:17:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:17:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.5706 192.168.0.1:0/77630305", 2023-10-16T08:17:59.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/77630305", 2023-10-16T08:17:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:17:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 18, 2023-10-16T08:17:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 39, 2023-10-16T08:17:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:17:59.545 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:17:59.546 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1" 2023-10-16T08:17:59.546 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:17:59.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:59.553 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:17:59.554 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:59.564 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:17:59.564 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:59.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:17:59.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:17:59.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-10-16T08:17:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 Processor -- start 2023-10-16T08:17:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 -- start start 2023-10-16T08:17:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ce610 0x7f77340cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f77340d1590 0x7f77340cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340d04f0 con 0x7f773415ffb0 2023-10-16T08:17:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77340d2000 con 0x7f77340d1590 2023-10-16T08:17:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f773a2dc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340d07d0 con 0x7f77340ce610 2023-10-16T08:17:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f77392da640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f773415ffb0 0x7f77340cdf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60822/0 (socket says 172.21.15.32:60822) 2023-10-16T08:17:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f77392da640 1 -- 172.21.15.32:0/1416096928 learned_addr learned my addr 172.21.15.32:0/1416096928 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:17:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f7738ad9640 1 --2- 172.21.15.32:0/1416096928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ce610 0x7f77340cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f7739adb640 1 --2- 172.21.15.32:0/1416096928 >> v2:172.21.15.186:3300/0 conn(0x7f77340d1590 0x7f77340cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f772bfff640 1 -- 172.21.15.32:0/1416096928 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521378689 0 0) 0x7f77340d04f0 con 0x7f773415ffb0 2023-10-16T08:17:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f772bfff640 1 -- 172.21.15.32:0/1416096928 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7720003530 con 0x7f773415ffb0 2023-10-16T08:17:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f7738ad9640 1 -- 172.21.15.32:0/1416096928 >> v2:172.21.15.186:3300/0 conn(0x7f77340d1590 msgr2=0x7f77340cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:17:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f7738ad9640 1 --2- 172.21.15.32:0/1416096928 >> v2:172.21.15.186:3300/0 conn(0x7f77340d1590 0x7f77340cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f7738ad9640 1 -- 172.21.15.32:0/1416096928 >> v1:172.21.15.32:6789/0 conn(0x7f773415ffb0 legacy=0x7f77340cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.813+0000 7f7738ad9640 1 -- 172.21.15.32:0/1416096928 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340d0ab0 con 0x7f77340ce610 2023-10-16T08:17:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f7738ad9640 1 --2- 172.21.15.32:0/1416096928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ce610 0x7f77340cf090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f77240097c0 tx=0x7f772401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326c2a4f6ceba080 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f772bfff640 1 -- 172.21.15.32:0/1416096928 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240ab600 con 0x7f77340ce610 2023-10-16T08:17:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f772bfff640 1 -- 172.21.15.32:0/1416096928 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77240a7050 con 0x7f77340ce610 2023-10-16T08:17:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f772bfff640 1 -- 172.21.15.32:0/1416096928 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240ab980 con 0x7f77340ce610 2023-10-16T08:17:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 -- 172.21.15.32:0/1416096928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ce610 msgr2=0x7f77340cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 --2- 172.21.15.32:0/1416096928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ce610 0x7f77340cf090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f77240097c0 tx=0x7f772401c480 comp rx=0 tx=0).stop 2023-10-16T08:17:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 -- 172.21.15.32:0/1416096928 shutdown_connections 2023-10-16T08:17:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 --2- 172.21.15.32:0/1416096928 >> v2:172.21.15.186:3300/0 conn(0x7f77340d1590 0x7f77340cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 --2- 172.21.15.32:0/1416096928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f77340ce610 0x7f77340cf090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:59.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 -- 172.21.15.32:0/1416096928 >> 172.21.15.32:0/1416096928 conn(0x7f77340d27a0 msgr2=0x7f77340d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:17:59.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 -- 172.21.15.32:0/1416096928 shutdown_connections 2023-10-16T08:17:59.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.817+0000 7f773a2dc640 1 -- 172.21.15.32:0/1416096928 wait complete. 2023-10-16T08:17:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 Processor -- start 2023-10-16T08:17:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- start start 2023-10-16T08:17:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 0x7f77340cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77340d1590 0x7f77340c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:59.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415ffb0 0x7f773415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340cc070 con 0x7f77340ce610 2023-10-16T08:17:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773415bd40 con 0x7f77340d1590 2023-10-16T08:17:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340ca280 con 0x7f773415ffb0 2023-10-16T08:17:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 0x7f77340cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 0x7f77340cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50876/0 (socket says 172.21.15.32:50876) 2023-10-16T08:17:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 -- 172.21.15.32:0/2211105277 learned_addr learned my addr 172.21.15.32:0/2211105277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:17:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f7738ad9640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77340d1590 0x7f77340c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f7739adb640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415ffb0 0x7f773415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 -- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415ffb0 msgr2=0x7f773415aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415ffb0 0x7f773415aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 -- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77340d1590 msgr2=0x7f77340c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:17:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77340d1590 0x7f77340c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:17:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 -- 172.21.15.32:0/2211105277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009420 con 0x7f77340ce610 2023-10-16T08:17:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f77392da640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 0x7f77340cbb30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f773000c450 tx=0x7f773000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730007480 con 0x7f77340ce610 2023-10-16T08:17:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77300075e0 con 0x7f77340ce610 2023-10-16T08:17:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300059d0 con 0x7f77340ce610 2023-10-16T08:17:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77340cb0a0 con 0x7f77340ce610 2023-10-16T08:17:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77340ca770 con 0x7f77340ce610 2023-10-16T08:17:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.821+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773404e9c0 con 0x7f77340ce610 2023-10-16T08:17:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.825+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7730005b30 con 0x7f77340ce610 2023-10-16T08:17:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.825+0000 7f7729ffb640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77180664d0 0x7f7718068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:17:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.825+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f77300835c0 con 0x7f77340ce610 2023-10-16T08:17:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.825+0000 7f7738ad9640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77180664d0 0x7f7718068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:17:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.825+0000 7f7738ad9640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77180664d0 0x7f7718068990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77240ad000 tx=0x7f7724008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:17:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.829+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f77300542f0 con 0x7f77340ce610 2023-10-16T08:17:59.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:17:59.985+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f773406caf0 con 0x7f77340ce610 2023-10-16T08:18:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.757+0000 7f7729ffb640 1 -- 172.21.15.32:0/2211105277 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v40) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7730017020 con 0x7f77340ce610 2023-10-16T08:18:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'data_new' created 2023-10-16T08:18:00.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77180664d0 msgr2=0x7f7718068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:00.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77180664d0 0x7f7718068990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77240ad000 tx=0x7f7724008040 comp rx=0 tx=0).stop 2023-10-16T08:18:00.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 msgr2=0x7f77340cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:00.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 0x7f77340cbb30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f773000c450 tx=0x7f773000c910 comp rx=0 tx=0).stop 2023-10-16T08:18:00.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 shutdown_connections 2023-10-16T08:18:00.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f77180664d0 0x7f7718068990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f773415ffb0 0x7f773415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77340d1590 0x7f77340c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 --2- 172.21.15.32:0/2211105277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77340ce610 0x7f77340cbb30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 >> 172.21.15.32:0/2211105277 conn(0x7f77340d27a0 msgr2=0x7f77340b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 shutdown_connections 2023-10-16T08:18:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.765+0000 7f773a2dc640 1 -- 172.21.15.32:0/2211105277 wait complete. 2023-10-16T08:18:00.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool testcephfs1 data_new 2023-10-16T08:18:00.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.941+0000 7f38987a1640 1 Processor -- start 2023-10-16T08:18:00.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.941+0000 7f38987a1640 1 -- start start 2023-10-16T08:18:00.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.941+0000 7f38987a1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38900d0210 0x7f38900d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:00.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f38987a1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f38900b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:00.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f38987a1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38900d1670 con 0x7f389015cba0 2023-10-16T08:18:00.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f38987a1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38900d1460 con 0x7f38900d0210 2023-10-16T08:18:00.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f38987a1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900d1950 con 0x7f38900b84e0 2023-10-16T08:18:00.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38900d0210 0x7f38900d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:00.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38900d0210 0x7f38900d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50882/0 (socket says 172.21.15.32:50882) 2023-10-16T08:18:00.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 -- 172.21.15.32:0/30302819 learned_addr learned my addr 172.21.15.32:0/30302819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:00.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3896d3e640 1 --2- 172.21.15.32:0/30302819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f38900b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:00.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 -- 172.21.15.32:0/30302819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 msgr2=0x7f38900b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:00.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 --2- 172.21.15.32:0/30302819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f38900b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 -- 172.21.15.32:0/30302819 >> v1:172.21.15.186:6789/0 conn(0x7f389015cba0 legacy=0x7f38900cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:00.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 -- 172.21.15.32:0/30302819 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900d1c30 con 0x7f38900d0210 2023-10-16T08:18:00.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f3895d3c640 1 --2- 172.21.15.32:0/30302819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f38900d0f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3880009870 tx=0x7f388001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe42f404423cadbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:00.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f389553b640 1 -- 172.21.15.32:0/30302819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800a9400 con 0x7f38900d0210 2023-10-16T08:18:00.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f389553b640 1 -- 172.21.15.32:0/30302819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38800a9560 con 0x7f38900d0210 2023-10-16T08:18:00.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.945+0000 7f389553b640 1 -- 172.21.15.32:0/30302819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800a98f0 con 0x7f38900d0210 2023-10-16T08:18:00.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- 172.21.15.32:0/30302819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 msgr2=0x7f38900d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:00.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 --2- 172.21.15.32:0/30302819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f38900d0f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3880009870 tx=0x7f388001c540 comp rx=0 tx=0).stop 2023-10-16T08:18:00.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- 172.21.15.32:0/30302819 shutdown_connections 2023-10-16T08:18:00.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 --2- 172.21.15.32:0/30302819 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f38900b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 --2- 172.21.15.32:0/30302819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f38900d0f20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- 172.21.15.32:0/30302819 >> 172.21.15.32:0/30302819 conn(0x7f38900719c0 msgr2=0x7f3890071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- 172.21.15.32:0/30302819 shutdown_connections 2023-10-16T08:18:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- 172.21.15.32:0/30302819 wait complete. 2023-10-16T08:18:00.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 Processor -- start 2023-10-16T08:18:00.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- start start 2023-10-16T08:18:00.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f3890156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:00.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f3890150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:00.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f389015cba0 0x7f3890157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:00.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890156fa0 con 0x7f38900d0210 2023-10-16T08:18:00.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38901561a0 con 0x7f389015cba0 2023-10-16T08:18:00.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f38987a1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38901507a0 con 0x7f38900b84e0 2023-10-16T08:18:00.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f3895d3c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f3890150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:00.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f3895d3c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f3890150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50898/0 (socket says 172.21.15.32:50898) 2023-10-16T08:18:00.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f3895d3c640 1 -- 172.21.15.32:0/3662940673 learned_addr learned my addr 172.21.15.32:0/3662940673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:00.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.949+0000 7f389653d640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f3890156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:00.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3896d3e640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f389015cba0 0x7f3890157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:00.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3895d3c640 1 -- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 msgr2=0x7f3890156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:00.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3895d3c640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f3890156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3895d3c640 1 -- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f389015cba0 msgr2=0x7f3890157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:00.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3895d3c640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f389015cba0 0x7f3890157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:00.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3895d3c640 1 -- 172.21.15.32:0/3662940673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38800094d0 con 0x7f38900d0210 2023-10-16T08:18:00.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f389653d640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f3890156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:00.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3896d3e640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f389015cba0 0x7f3890157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:00.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3895d3c640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f3890150260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f388001abf0 tx=0x7f388001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:00.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800a9400 con 0x7f38900d0210 2023-10-16T08:18:00.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38800b1800 con 0x7f38900d0210 2023-10-16T08:18:00.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800b1bb0 con 0x7f38900d0210 2023-10-16T08:18:00.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3890158e80 con 0x7f38900d0210 2023-10-16T08:18:00.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3890155b50 con 0x7f38900d0210 2023-10-16T08:18:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3890156350 con 0x7f38900d0210 2023-10-16T08:18:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f38800b1d10 con 0x7f38900d0210 2023-10-16T08:18:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3886ffd640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38740664a0 0x7f3874068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:00.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.953+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6412+0+0 (secure 0 0 0) 0x7f388011e8c0 con 0x7f38900d0210 2023-10-16T08:18:00.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.957+0000 7f389653d640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38740664a0 0x7f3874068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:00.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.957+0000 7f389653d640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38740664a0 0x7f3874068960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f388c000f70 tx=0x7f388c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:00.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:00.961+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3890156350 con 0x7f38900d0210 2023-10-16T08:18:01.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.153+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "testcephfs1", "pool": "data_new"} v 0) v1 -- 0x7f3890156350 con 0x7f38900d0210 2023-10-16T08:18:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f3886ffd640 1 -- 172.21.15.32:0/3662940673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "testcephfs1", "pool": "data_new"}]=0 added data pool 7 to fsmap v19) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3890156350 con 0x7f38900d0210 2023-10-16T08:18:01.767 INFO:teuthology.orchestra.run.smithi032.stderr:added data pool 7 to fsmap 2023-10-16T08:18:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38740664a0 msgr2=0x7f3874068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f38987a1640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38740664a0 0x7f3874068960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f388c000f70 tx=0x7f388c009040 comp rx=0 tx=0).stop 2023-10-16T08:18:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 msgr2=0x7f3890150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:01.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f38987a1640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f3890150260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f388001abf0 tx=0x7f388001ad80 comp rx=0 tx=0).stop 2023-10-16T08:18:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 shutdown_connections 2023-10-16T08:18:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.761+0000 7f38987a1640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f389015cba0 0x7f3890157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.765+0000 7f38987a1640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38900d0210 0x7f3890150260 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.765+0000 7f38987a1640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38740664a0 0x7f3874068960 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.765+0000 7f38987a1640 1 --2- 172.21.15.32:0/3662940673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38900b84e0 0x7f3890156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.765+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 >> 172.21.15.32:0/3662940673 conn(0x7f38900719c0 msgr2=0x7f3890154550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.765+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 shutdown_connections 2023-10-16T08:18:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.765+0000 7f38987a1640 1 -- 172.21.15.32:0/3662940673 wait complete. 2023-10-16T08:18:01.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:01.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 Processor -- start 2023-10-16T08:18:01.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 -- start start 2023-10-16T08:18:01.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd9e415ca70 0x7fd9e40d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9e40d3550 0x7fd9e40cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e40d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e40ce820 con 0x7fd9e40d3550 2023-10-16T08:18:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e40cfa40 con 0x7fd9e415ca70 2023-10-16T08:18:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9ea854640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40c65f0 con 0x7fd9e40cf5d0 2023-10-16T08:18:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9e40d3550 0x7fd9e40cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9e40d3550 0x7fd9e40cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50902/0 (socket says 172.21.15.32:50902) 2023-10-16T08:18:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e37fe640 1 -- 172.21.15.32:0/785725072 learned_addr learned my addr 172.21.15.32:0/785725072 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e8df1640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e40d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e3fff640 1 --2- 172.21.15.32:0/785725072 >> v2:172.21.15.186:3300/0 conn(0x7fd9e415ca70 0x7fd9e40d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e37fe640 1 -- 172.21.15.32:0/785725072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 msgr2=0x7fd9e40d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e37fe640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e40d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.929+0000 7fd9e37fe640 1 -- 172.21.15.32:0/785725072 >> v2:172.21.15.186:3300/0 conn(0x7fd9e415ca70 msgr2=0x7fd9e40d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e37fe640 1 --2- 172.21.15.32:0/785725072 >> v2:172.21.15.186:3300/0 conn(0x7fd9e415ca70 0x7fd9e40d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e37fe640 1 -- 172.21.15.32:0/785725072 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40cdf80 con 0x7fd9e40d3550 2023-10-16T08:18:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e3fff640 1 --2- 172.21.15.32:0/785725072 >> v2:172.21.15.186:3300/0 conn(0x7fd9e415ca70 0x7fd9e40d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e8df1640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e40d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e37fe640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e40d3550 0x7fd9e40cf090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd9e408a880 tx=0x7fd9d801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a545abfc6e8a45 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/785725072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80ab600 con 0x7fd9e40d3550 2023-10-16T08:18:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/785725072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d80a7050 con 0x7fd9e40d3550 2023-10-16T08:18:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/785725072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80ab980 con 0x7fd9e40d3550 2023-10-16T08:18:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 -- 172.21.15.32:0/785725072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e40d3550 msgr2=0x7fd9e40cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e40d3550 0x7fd9e40cf090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd9e408a880 tx=0x7fd9d801c210 comp rx=0 tx=0).stop 2023-10-16T08:18:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 -- 172.21.15.32:0/785725072 shutdown_connections 2023-10-16T08:18:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e40d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 --2- 172.21.15.32:0/785725072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e40d3550 0x7fd9e40cf090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 --2- 172.21.15.32:0/785725072 >> v2:172.21.15.186:3300/0 conn(0x7fd9e415ca70 0x7fd9e40d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 -- 172.21.15.32:0/785725072 >> 172.21.15.32:0/785725072 conn(0x7fd9e409bc50 msgr2=0x7fd9e40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 -- 172.21.15.32:0/785725072 shutdown_connections 2023-10-16T08:18:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.933+0000 7fd9ea854640 1 -- 172.21.15.32:0/785725072 wait complete. 2023-10-16T08:18:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 Processor -- start 2023-10-16T08:18:01.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 -- start start 2023-10-16T08:18:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40d3550 0x7fd9e4152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 0x7fd9e415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e415e500 con 0x7fd9e415ca70 2023-10-16T08:18:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e41603d0 con 0x7fd9e40d3550 2023-10-16T08:18:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e415eaa0 con 0x7fd9e40cf5d0 2023-10-16T08:18:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 0x7fd9e415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 0x7fd9e415ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50908/0 (socket says 172.21.15.32:50908) 2023-10-16T08:18:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 -- 172.21.15.32:0/1303095591 learned_addr learned my addr 172.21.15.32:0/1303095591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e3fff640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e37fe640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40d3550 0x7fd9e4152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 -- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 msgr2=0x7fd9e415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 -- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40d3550 msgr2=0x7fd9e4152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40d3550 0x7fd9e4152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 -- 172.21.15.32:0/1303095591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80094d0 con 0x7fd9e415ca70 2023-10-16T08:18:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e37fe640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40d3550 0x7fd9e4152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e8df1640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 0x7fd9e415ef30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00cd50 tx=0x7fd9dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00f680 con 0x7fd9e415ca70 2023-10-16T08:18:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc00f800 con 0x7fd9e415ca70 2023-10-16T08:18:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e415fe10 con 0x7fd9e415ca70 2023-10-16T08:18:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00fbb0 con 0x7fd9e415ca70 2023-10-16T08:18:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.937+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e415e730 con 0x7fd9e415ca70 2023-10-16T08:18:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.941+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e415ec10 con 0x7fd9e415ca70 2023-10-16T08:18:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.941+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9dc00b040 con 0x7fd9e415ca70 2023-10-16T08:18:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.941+0000 7fd9e0ff9640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd9d00664d0 0x7fd9d0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.941+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fd9dc084a90 con 0x7fd9e415ca70 2023-10-16T08:18:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.941+0000 7fd9e3fff640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd9d00664d0 0x7fd9d0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.945+0000 7fd9e3fff640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd9d00664d0 0x7fd9d0068990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4159930 tx=0x7fd9d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:01.945+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9dc051c10 con 0x7fd9e415ca70 2023-10-16T08:18:02.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.133+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9e4074010 con 0x7fd9e415ca70 2023-10-16T08:18:02.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9e0ff9640 1 -- 172.21.15.32:0/1303095591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fd9dc055230 con 0x7fd9e415ca70 2023-10-16T08:18:02.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:02.143 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:18:01.158878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:18:02.143 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2023-10-16T08:18:02.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd9d00664d0 msgr2=0x7fd9d0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd9d00664d0 0x7fd9d0068990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4159930 tx=0x7fd9d4008040 comp rx=0 tx=0).stop 2023-10-16T08:18:02.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 msgr2=0x7fd9e415ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 0x7fd9e415ef30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00cd50 tx=0x7fd9dc00a650 comp rx=0 tx=0).stop 2023-10-16T08:18:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 shutdown_connections 2023-10-16T08:18:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9e415ca70 0x7fd9e415ef30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40d3550 0x7fd9e4152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd9d00664d0 0x7fd9d0068990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 --2- 172.21.15.32:0/1303095591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd9e40cf5d0 0x7fd9e415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 >> 172.21.15.32:0/1303095591 conn(0x7fd9e409bc50 msgr2=0x7fd9e40cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:02.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 shutdown_connections 2023-10-16T08:18:02.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.137+0000 7fd9ea854640 1 -- 172.21.15.32:0/1303095591 wait complete. 2023-10-16T08:18:02.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 Processor -- start 2023-10-16T08:18:02.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 -- start start 2023-10-16T08:18:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b780720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9b780b00e0 0x7f9b780b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b780b0a70 con 0x7f9b781591e0 2023-10-16T08:18:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b78071020 con 0x7f9b780b00e0 2023-10-16T08:18:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7eb94640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b780c8e50 con 0x7f9b78071c70 2023-10-16T08:18:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7c930640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b781591e0 0x7f9b780c9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60824/0 (socket says 172.21.15.32:60824) 2023-10-16T08:18:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7c930640 1 -- 172.21.15.32:0/2700952002 learned_addr learned my addr 172.21.15.32:0/2700952002 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b780720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7d131640 1 --2- 172.21.15.32:0/2700952002 >> v2:172.21.15.186:3300/0 conn(0x7f9b780b00e0 0x7f9b780b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345134136 0 0) 0x7f9b6c001120 con 0x7f9b781591e0 2023-10-16T08:18:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b68003530 con 0x7f9b781591e0 2023-10-16T08:18:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4042105607 0 0) 0x7f9b68003530 con 0x7f9b781591e0 2023-10-16T08:18:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 msgr2=0x7f9b780720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b780720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 >> v2:172.21.15.186:3300/0 conn(0x7f9b780b00e0 msgr2=0x7f9b780b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 --2- 172.21.15.32:0/2700952002 >> v2:172.21.15.186:3300/0 conn(0x7f9b780b00e0 0x7f9b780b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b780c9130 con 0x7f9b781591e0 2023-10-16T08:18:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f9b6c001810 con 0x7f9b781591e0 2023-10-16T08:18:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 >> v1:172.21.15.32:6789/0 conn(0x7f9b781591e0 legacy=0x7f9b780c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68003980 0x7f9b68003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b68005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b78071020 con 0x7f9b680047d0 2023-10-16T08:18:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b780c8e50 con 0x7f9b68003980 2023-10-16T08:18:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6800b990 con 0x7f9b78071c70 2023-10-16T08:18:02.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.305+0000 7f9b7c930640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68003980 0x7f9b68003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b7d131640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b68005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b77fff640 1 -- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 msgr2=0x7f9b68005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b68005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b77fff640 1 -- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68003980 msgr2=0x7f9b68003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68003980 0x7f9b68003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b77fff640 1 -- 172.21.15.32:0/2700952002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68003530 con 0x7f9b680047d0 2023-10-16T08:18:02.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b7d131640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b78071c70 0x7f9b68005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b68004c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400bdd0 tx=0x7f9b6401ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece381d2e536451e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640a76f0 con 0x7f9b680047d0 2023-10-16T08:18:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b640a7850 con 0x7f9b680047d0 2023-10-16T08:18:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b777fe640 1 -- 172.21.15.32:0/2700952002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640a7bc0 con 0x7f9b680047d0 2023-10-16T08:18:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2700952002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 msgr2=0x7f9b68004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2700952002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b68004c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400bdd0 tx=0x7f9b6401ec20 comp rx=0 tx=0).stop 2023-10-16T08:18:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.309+0000 7f9b7d131640 1 -- 172.21.15.32:0/2700952002 reap_dead start 2023-10-16T08:18:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2700952002 shutdown_connections 2023-10-16T08:18:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2700952002 >> 172.21.15.32:0/2700952002 conn(0x7f9b7806c190 msgr2=0x7f9b7806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2700952002 shutdown_connections 2023-10-16T08:18:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2700952002 wait complete. 2023-10-16T08:18:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 Processor -- start 2023-10-16T08:18:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- start start 2023-10-16T08:18:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b781595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b78157330 0x7f9b78146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b7815d010 0x7f9b7815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7815d9d0 con 0x7f9b680047d0 2023-10-16T08:18:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b780b1e00 con 0x7f9b78157330 2023-10-16T08:18:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7eb94640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b78159ba0 con 0x7f9b7815d010 2023-10-16T08:18:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b781595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b781595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50924/0 (socket says 172.21.15.32:50924) 2023-10-16T08:18:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 -- 172.21.15.32:0/2466335170 learned_addr learned my addr 172.21.15.32:0/2466335170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b78157330 0x7f9b78146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7d131640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b7815d010 0x7f9b7815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 -- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b7815d010 msgr2=0x7f9b7815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b7815d010 0x7f9b7815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 -- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b78157330 msgr2=0x7f9b78146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b78157330 0x7f9b78146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 -- 172.21.15.32:0/2466335170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78159e80 con 0x7f9b680047d0 2023-10-16T08:18:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b78157330 0x7f9b78146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.313+0000 7f9b7c930640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b781595c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005850 tx=0x7f9b6c0126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c014750 con 0x7f9b680047d0 2023-10-16T08:18:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0148d0 con 0x7f9b680047d0 2023-10-16T08:18:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c014c80 con 0x7f9b680047d0 2023-10-16T08:18:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7815a160 con 0x7f9b680047d0 2023-10-16T08:18:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b78146e80 con 0x7f9b680047d0 2023-10-16T08:18:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b780b1f70 con 0x7f9b680047d0 2023-10-16T08:18:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b6c0153e0 con 0x7f9b680047d0 2023-10-16T08:18:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.317+0000 7f9b757fa640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9b5c0664f0 0x7f9b5c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.321+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f9b6c086b40 con 0x7f9b680047d0 2023-10-16T08:18:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.321+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9b5c0664f0 0x7f9b5c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.321+0000 7f9b77fff640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9b5c0664f0 0x7f9b5c0689b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9b640af000 tx=0x7f9b6401d460 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.325+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b6c053d40 con 0x7f9b680047d0 2023-10-16T08:18:02.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.481+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b78074010 con 0x7f9b680047d0 2023-10-16T08:18:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.481+0000 7f9b757fa640 1 -- 172.21.15.32:0/2466335170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18591 (secure 0 0 0) 0x7f9b6c01c020 con 0x7f9b680047d0 2023-10-16T08:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:02.488 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":41,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:01.158629+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9b5c0664f0 msgr2=0x7f9b5c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9b5c0664f0 0x7f9b5c0689b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9b640af000 tx=0x7f9b6401d460 comp rx=0 tx=0).stop 2023-10-16T08:18:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 msgr2=0x7f9b781595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b781595c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005850 tx=0x7f9b6c0126f0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 shutdown_connections 2023-10-16T08:18:02.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9b5c0664f0 0x7f9b5c0689b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9b7815d010 0x7f9b7815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b78157330 0x7f9b78146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 --2- 172.21.15.32:0/2466335170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b680047d0 0x7f9b781595c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 >> 172.21.15.32:0/2466335170 conn(0x7f9b7806c190 msgr2=0x7f9b78154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:02.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 shutdown_connections 2023-10-16T08:18:02.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.485+0000 7f9b7eb94640 1 -- 172.21.15.32:0/2466335170 wait complete. 2023-10-16T08:18:02.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:02.511 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-10-16T08:18:02.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:02.530 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-10-16T08:18:02.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=testcephfs1_data,allow rw pool=data_new' 2023-10-16T08:18:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 Processor -- start 2023-10-16T08:18:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 -- start start 2023-10-16T08:18:02.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 0x7f482006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f482006a5a0 0x7f48200c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48200cd440 con 0x7f4820071120 2023-10-16T08:18:02.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482006ccd0 con 0x7f482006a5a0 2023-10-16T08:18:02.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4827594640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482006cfb0 con 0x7f482015cce0 2023-10-16T08:18:02.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4824b2f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4820071120 0x7f4820069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60830/0 (socket says 172.21.15.32:60830) 2023-10-16T08:18:02.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4824b2f640 1 -- 172.21.15.32:0/1977851128 learned_addr learned my addr 172.21.15.32:0/1977851128 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825b31640 1 --2- 172.21.15.32:0/1977851128 >> v2:172.21.15.186:3300/0 conn(0x7f482006a5a0 0x7f48200c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458517495 0 0) 0x7f48200cd440 con 0x7f4820071120 2023-10-16T08:18:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 0x7f482006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f480c003530 con 0x7f4820071120 2023-10-16T08:18:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862192716 0 0) 0x7f480c003530 con 0x7f4820071120 2023-10-16T08:18:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 msgr2=0x7f482006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 0x7f482006dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 >> v2:172.21.15.186:3300/0 conn(0x7f482006a5a0 msgr2=0x7f48200c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 --2- 172.21.15.32:0/1977851128 >> v2:172.21.15.186:3300/0 conn(0x7f482006a5a0 0x7f48200c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482006d290 con 0x7f4820071120 2023-10-16T08:18:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f4810001680 con 0x7f4820071120 2023-10-16T08:18:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 >> v1:172.21.15.32:6789/0 conn(0x7f4820071120 legacy=0x7f4820069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f480c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 0x7f480c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f480c004d10 0x7f480c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482006ccd0 con 0x7f480c003980 2023-10-16T08:18:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482006cfb0 con 0x7f480c004d10 2023-10-16T08:18:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c00b990 con 0x7f482015cce0 2023-10-16T08:18:02.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f480c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4824b2f640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 0x7f480c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825b31640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f480c004d10 0x7f480c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 -- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 msgr2=0x7f480c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f482015cce0 0x7f480c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 -- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f480c004d10 msgr2=0x7f480c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:02.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f480c004d10 0x7f480c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.785+0000 7f4825330640 1 -- 172.21.15.32:0/1977851128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c003530 con 0x7f480c003980 2023-10-16T08:18:02.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4825330640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f480c003da0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ac0 tx=0x7f481c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43becaba371e852 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:02.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0a5510 con 0x7f480c003980 2023-10-16T08:18:02.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c0a5670 con 0x7f480c003980 2023-10-16T08:18:02.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4817fff640 1 -- 172.21.15.32:0/1977851128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0a5a00 con 0x7f480c003980 2023-10-16T08:18:02.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4827594640 1 -- 172.21.15.32:0/1977851128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 msgr2=0x7f480c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4827594640 1 --2- 172.21.15.32:0/1977851128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f480c003da0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ac0 tx=0x7f481c01ca30 comp rx=0 tx=0).stop 2023-10-16T08:18:02.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4825b31640 1 -- 172.21.15.32:0/1977851128 reap_dead start 2023-10-16T08:18:02.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4827594640 1 -- 172.21.15.32:0/1977851128 shutdown_connections 2023-10-16T08:18:02.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4827594640 1 -- 172.21.15.32:0/1977851128 >> 172.21.15.32:0/1977851128 conn(0x7f48200cb660 msgr2=0x7f48200cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4827594640 1 -- 172.21.15.32:0/1977851128 shutdown_connections 2023-10-16T08:18:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.789+0000 7f4827594640 1 -- 172.21.15.32:0/1977851128 wait complete. 2023-10-16T08:18:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 Processor -- start 2023-10-16T08:18:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 -- start start 2023-10-16T08:18:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f482015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48200c7df0 0x7f48200c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48200c36b0 0x7f48200c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c5060 con 0x7f480c003980 2023-10-16T08:18:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200b2dd0 con 0x7f48200c7df0 2023-10-16T08:18:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c87b0 con 0x7f48200c36b0 2023-10-16T08:18:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f482015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f482015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50940/0 (socket says 172.21.15.32:50940) 2023-10-16T08:18:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 -- 172.21.15.32:0/628392311 learned_addr learned my addr 172.21.15.32:0/628392311 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4824b2f640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48200c7df0 0x7f48200c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825b31640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48200c36b0 0x7f48200c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 -- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48200c36b0 msgr2=0x7f48200c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48200c36b0 0x7f48200c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 -- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48200c7df0 msgr2=0x7f48200c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48200c7df0 0x7f48200c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 -- 172.21.15.32:0/628392311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200c43e0 con 0x7f480c003980 2023-10-16T08:18:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4825330640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f482015a990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ac0 tx=0x7f480c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0ad850 con 0x7f480c003980 2023-10-16T08:18:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c0ad9b0 con 0x7f480c003980 2023-10-16T08:18:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0add60 con 0x7f480c003980 2023-10-16T08:18:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48200c46c0 con 0x7f480c003980 2023-10-16T08:18:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.793+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48200c4c80 con 0x7f480c003980 2023-10-16T08:18:02.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.797+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482004e170 con 0x7f480c003980 2023-10-16T08:18:02.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.797+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f481c0b4040 con 0x7f480c003980 2023-10-16T08:18:02.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.797+0000 7f4815ffb640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48040664a0 0x7f4804068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:02.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.797+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f481c0bc030 con 0x7f480c003980 2023-10-16T08:18:02.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.797+0000 7f4824b2f640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48040664a0 0x7f4804068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:02.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.797+0000 7f4824b2f640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48040664a0 0x7f4804068960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4810001ec0 tx=0x7f4810009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:02.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:02.801+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f481c0eb7d0 con 0x7f480c003980 2023-10-16T08:18:03.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.013+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f48200c5480 con 0x7f480c003980 2023-10-16T08:18:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4815ffb640 1 -- 172.21.15.32:0/628392311 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v10) v1 ==== 215+0+172 (secure 0 0 0) 0x7f481c0eedf0 con 0x7f480c003980 2023-10-16T08:18:03.023 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:18:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:18:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow rw" 2023-10-16T08:18:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:18:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw pool=testcephfs1_data,allow rw pool=data_new" 2023-10-16T08:18:03.025 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:18:03.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48040664a0 msgr2=0x7f4804068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48040664a0 0x7f4804068960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4810001ec0 tx=0x7f4810009040 comp rx=0 tx=0).stop 2023-10-16T08:18:03.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 msgr2=0x7f482015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f482015a990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ac0 tx=0x7f480c005800 comp rx=0 tx=0).stop 2023-10-16T08:18:03.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 shutdown_connections 2023-10-16T08:18:03.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f48040664a0 0x7f4804068960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f48200c36b0 0x7f48200c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48200c7df0 0x7f48200c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 --2- 172.21.15.32:0/628392311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c003980 0x7f482015a990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 >> 172.21.15.32:0/628392311 conn(0x7f48200cb660 msgr2=0x7f4820069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:03.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 shutdown_connections 2023-10-16T08:18:03.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.017+0000 7f4827594640 1 -- 172.21.15.32:0/628392311 wait complete. 2023-10-16T08:18:03.043 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:03.044 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:03.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f617630f640 1 Processor -- start 2023-10-16T08:18:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f617630f640 1 -- start start 2023-10-16T08:18:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f617630f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61700cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f617630f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700cfa30 con 0x7f617015ca70 2023-10-16T08:18:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f617630f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700c65f0 con 0x7f61700c6ad0 2023-10-16T08:18:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f617630f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700ce820 con 0x7f61700cf090 2023-10-16T08:18:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f617015ca70 0x7f61700d3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59132/0 (socket says 172.21.15.32:59132) 2023-10-16T08:18:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616ffff640 1 -- 172.21.15.32:0/3885806070 learned_addr learned my addr 172.21.15.32:0/3885806070 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558698627 0 0) 0x7f61700cfa30 con 0x7f617015ca70 2023-10-16T08:18:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6158003530 con 0x7f617015ca70 2023-10-16T08:18:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616f7fe640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61700cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 424270642 0 0) 0x7f6158003530 con 0x7f617015ca70 2023-10-16T08:18:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 msgr2=0x7f61700cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61700cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 >> v1:172.21.15.186:6789/0 conn(0x7f61700c6ad0 legacy=0x7f61700c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700cdf80 con 0x7f617015ca70 2023-10-16T08:18:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f6160001800 con 0x7f617015ca70 2023-10-16T08:18:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 >> v1:172.21.15.32:6789/0 conn(0x7f617015ca70 legacy=0x7f61700d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61580039b0 0x7f6158003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6158004d10 0x7f6158005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700ce820 con 0x7f6158004d10 2023-10-16T08:18:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615800ba40 con 0x7f61580039b0 2023-10-16T08:18:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615800bc00 con 0x7f61700cf090 2023-10-16T08:18:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f61748ac640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6158004d10 0x7f6158005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616f7fe640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.237+0000 7f616ffff640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61580039b0 0x7f6158003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 -- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 msgr2=0x7f61580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 -- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61580039b0 msgr2=0x7f6158003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61580039b0 0x7f6158003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 -- 172.21.15.32:0/3885806070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158003530 con 0x7f6158004d10 2023-10-16T08:18:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f616f7fe640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f61700cf090 0x7f61580047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6158004d10 0x7f6158005190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6168001e40 tx=0x7f616801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a532f594d468b111 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168003b80 con 0x7f6158004d10 2023-10-16T08:18:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168003ce0 con 0x7f6158004d10 2023-10-16T08:18:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f616effd640 1 -- 172.21.15.32:0/3885806070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680052c0 con 0x7f6158004d10 2023-10-16T08:18:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f617630f640 1 -- 172.21.15.32:0/3885806070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6158004d10 msgr2=0x7f6158005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f617630f640 1 --2- 172.21.15.32:0/3885806070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6158004d10 0x7f6158005190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6168001e40 tx=0x7f616801c740 comp rx=0 tx=0).stop 2023-10-16T08:18:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f61748ac640 1 -- 172.21.15.32:0/3885806070 reap_dead start 2023-10-16T08:18:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f617630f640 1 -- 172.21.15.32:0/3885806070 shutdown_connections 2023-10-16T08:18:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f617630f640 1 -- 172.21.15.32:0/3885806070 >> 172.21.15.32:0/3885806070 conn(0x7f617009bc50 msgr2=0x7f61700d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.241+0000 7f617630f640 1 -- 172.21.15.32:0/3885806070 shutdown_connections 2023-10-16T08:18:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 -- 172.21.15.32:0/3885806070 wait complete. 2023-10-16T08:18:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 Processor -- start 2023-10-16T08:18:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 -- start start 2023-10-16T08:18:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6158004d10 0x7f617014dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f617015f8f0 0x7f617015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 0x7f61700c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700c8f40 con 0x7f61700c8580 2023-10-16T08:18:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700b83b0 con 0x7f617015f8f0 2023-10-16T08:18:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f617630f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617014e160 con 0x7f6158004d10 2023-10-16T08:18:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 0x7f61700c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 0x7f61700c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40364/0 (socket says 172.21.15.32:40364) 2023-10-16T08:18:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 -- 172.21.15.32:0/1093203295 learned_addr learned my addr 172.21.15.32:0/1093203295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f616ffff640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6158004d10 0x7f617014dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f616f7fe640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f617015f8f0 0x7f617015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 -- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6158004d10 msgr2=0x7f617014dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6158004d10 0x7f617014dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 -- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f617015f8f0 msgr2=0x7f617015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f617015f8f0 0x7f617015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 -- 172.21.15.32:0/1093203295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700b77a0 con 0x7f61700c8580 2023-10-16T08:18:03.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.245+0000 7f61748ac640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 0x7f61700c8a00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f616801c780 tx=0x7f6158005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168005230 con 0x7f61700c8580 2023-10-16T08:18:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168005390 con 0x7f61700c8580 2023-10-16T08:18:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168005740 con 0x7f61700c8580 2023-10-16T08:18:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700b7a80 con 0x7f61700c8580 2023-10-16T08:18:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61700b8040 con 0x7f61700c8580 2023-10-16T08:18:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61700cfda0 con 0x7f61700c8580 2023-10-16T08:18:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61680a6440 con 0x7f61700c8580 2023-10-16T08:18:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616cff9640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f615c0664a0 0x7f615c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f616811d4d0 con 0x7f61700c8580 2023-10-16T08:18:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.249+0000 7f616ffff640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f615c0664a0 0x7f615c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.253+0000 7f616ffff640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f615c0664a0 0x7f615c068960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6160000f00 tx=0x7f6160009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.253+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61680ea5f0 con 0x7f61700c8580 2023-10-16T08:18:03.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6170074010 con 0x7f61700c8580 2023-10-16T08:18:03.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f616cff9640 1 -- 172.21.15.32:0/1093203295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7f61680ba090 con 0x7f61700c8580 2023-10-16T08:18:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:03.419 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f615c0664a0 msgr2=0x7f615c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f615c0664a0 0x7f615c068960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6160000f00 tx=0x7f6160009040 comp rx=0 tx=0).stop 2023-10-16T08:18:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 msgr2=0x7f61700c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 0x7f61700c8a00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f616801c780 tx=0x7f6158005800 comp rx=0 tx=0).stop 2023-10-16T08:18:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 shutdown_connections 2023-10-16T08:18:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61700c8580 0x7f61700c8a00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f617015f8f0 0x7f617015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f615c0664a0 0x7f615c068960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 --2- 172.21.15.32:0/1093203295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6158004d10 0x7f617014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 >> 172.21.15.32:0/1093203295 conn(0x7f617009bc50 msgr2=0x7f61700c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 shutdown_connections 2023-10-16T08:18:03.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.413+0000 7f617630f640 1 -- 172.21.15.32:0/1093203295 wait complete. 2023-10-16T08:18:03.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:03.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.581+0000 7fbf1b90e640 1 Processor -- start 2023-10-16T08:18:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.581+0000 7fbf1b90e640 1 -- start start 2023-10-16T08:18:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf1b90e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbf1415cba0 0x7fbf140d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf1b90e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf140cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf1b90e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf140d0040 0x7fbf140d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf1b90e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf140d1460 con 0x7fbf140d0040 2023-10-16T08:18:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf1b90e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf140d04b0 con 0x7fbf1415cba0 2023-10-16T08:18:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf1b90e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140d1670 con 0x7fbf140b84e0 2023-10-16T08:18:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf140d0040 0x7fbf140d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf140d0040 0x7fbf140d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40366/0 (socket says 172.21.15.32:40366) 2023-10-16T08:18:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 -- 172.21.15.32:0/2809924242 learned_addr learned my addr 172.21.15.32:0/2809924242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf196aa640 1 --2- 172.21.15.32:0/2809924242 >> v2:172.21.15.186:3300/0 conn(0x7fbf1415cba0 0x7fbf140d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf18ea9640 1 --2- 172.21.15.32:0/2809924242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf140cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 -- 172.21.15.32:0/2809924242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 msgr2=0x7fbf140cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 --2- 172.21.15.32:0/2809924242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf140cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 -- 172.21.15.32:0/2809924242 >> v2:172.21.15.186:3300/0 conn(0x7fbf1415cba0 msgr2=0x7fbf140d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:03.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 --2- 172.21.15.32:0/2809924242 >> v2:172.21.15.186:3300/0 conn(0x7fbf1415cba0 0x7fbf140d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 -- 172.21.15.32:0/2809924242 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140d1950 con 0x7fbf140d0040 2023-10-16T08:18:03.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf196aa640 1 --2- 172.21.15.32:0/2809924242 >> v2:172.21.15.186:3300/0 conn(0x7fbf1415cba0 0x7fbf140d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:03.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf19eab640 1 --2- 172.21.15.32:0/2809924242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf140d0040 0x7fbf140d2ae0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b4890 tx=0x7fbf0c01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6443b357a44377 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.585+0000 7fbf0bfff640 1 -- 172.21.15.32:0/2809924242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0a5590 con 0x7fbf140d0040 2023-10-16T08:18:03.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf0bfff640 1 -- 172.21.15.32:0/2809924242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0a56f0 con 0x7fbf140d0040 2023-10-16T08:18:03.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf0bfff640 1 -- 172.21.15.32:0/2809924242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0a5a80 con 0x7fbf140d0040 2023-10-16T08:18:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2809924242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf140d0040 msgr2=0x7fbf140d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2809924242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf140d0040 0x7fbf140d2ae0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b4890 tx=0x7fbf0c01c710 comp rx=0 tx=0).stop 2023-10-16T08:18:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2809924242 shutdown_connections 2023-10-16T08:18:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2809924242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf140d0040 0x7fbf140d2ae0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2809924242 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf140cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2809924242 >> v2:172.21.15.186:3300/0 conn(0x7fbf1415cba0 0x7fbf140d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2809924242 >> 172.21.15.32:0/2809924242 conn(0x7fbf140719c0 msgr2=0x7fbf14071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2809924242 shutdown_connections 2023-10-16T08:18:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2809924242 wait complete. 2023-10-16T08:18:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 Processor -- start 2023-10-16T08:18:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- start start 2023-10-16T08:18:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf1414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf140d0040 0x7fbf14154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 0x7fbf141545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14159480 con 0x7fbf1415cba0 2023-10-16T08:18:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14159760 con 0x7fbf140d0040 2023-10-16T08:18:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf1b90e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1414f3d0 con 0x7fbf140b84e0 2023-10-16T08:18:03.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf19eab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 0x7fbf141545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf19eab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 0x7fbf141545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40370/0 (socket says 172.21.15.32:40370) 2023-10-16T08:18:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf19eab640 1 -- 172.21.15.32:0/2468579546 learned_addr learned my addr 172.21.15.32:0/2468579546 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf18ea9640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf140d0040 0x7fbf14154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.589+0000 7fbf196aa640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf1414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf19eab640 1 -- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 msgr2=0x7fbf1414e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf19eab640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf1414e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf19eab640 1 -- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf140d0040 msgr2=0x7fbf14154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf19eab640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf140d0040 0x7fbf14154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf19eab640 1 -- 172.21.15.32:0/2468579546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c009b30 con 0x7fbf1415cba0 2023-10-16T08:18:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf19eab640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 0x7fbf141545d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0af040 tx=0x7fbf0c0194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0b8670 con 0x7fbf1415cba0 2023-10-16T08:18:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0b87f0 con 0x7fbf1415cba0 2023-10-16T08:18:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0b8ba0 con 0x7fbf1415cba0 2023-10-16T08:18:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1414f6b0 con 0x7fbf1415cba0 2023-10-16T08:18:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1414fc70 con 0x7fbf1415cba0 2023-10-16T08:18:03.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1404e170 con 0x7fbf1415cba0 2023-10-16T08:18:03.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.593+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbf0c0b8d00 con 0x7fbf1415cba0 2023-10-16T08:18:03.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.597+0000 7fbf09ffb640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbef80664a0 0x7fbef8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.597+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fbf0c11f420 con 0x7fbf1415cba0 2023-10-16T08:18:03.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.597+0000 7fbf196aa640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbef80664a0 0x7fbef8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.597+0000 7fbf196aa640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbef80664a0 0x7fbef8068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf100045f0 tx=0x7fbf10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.597+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbf0c0ec690 con 0x7fbf1415cba0 2023-10-16T08:18:03.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.757+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf140c94b0 con 0x7fbf1415cba0 2023-10-16T08:18:03.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.757+0000 7fbf09ffb640 1 -- 172.21.15.32:0/2468579546 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf0c009510 con 0x7fbf1415cba0 2023-10-16T08:18:03.765 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:03.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbef80664a0 msgr2=0x7fbef8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbef80664a0 0x7fbef8068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf100045f0 tx=0x7fbf10008040 comp rx=0 tx=0).stop 2023-10-16T08:18:03.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 msgr2=0x7fbf141545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 0x7fbf141545d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0af040 tx=0x7fbf0c0194b0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 shutdown_connections 2023-10-16T08:18:03.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1415cba0 0x7fbf141545d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf140d0040 0x7fbf14154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbef80664a0 0x7fbef8068960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 --2- 172.21.15.32:0/2468579546 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbf140b84e0 0x7fbf1414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 >> 172.21.15.32:0/2468579546 conn(0x7fbf140719c0 msgr2=0x7fbf140b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:03.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 shutdown_connections 2023-10-16T08:18:03.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.761+0000 7fbf1b90e640 1 -- 172.21.15.32:0/2468579546 wait complete. 2023-10-16T08:18:03.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:03.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.925+0000 7f96455d9640 1 Processor -- start 2023-10-16T08:18:03.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.925+0000 7f96455d9640 1 -- start start 2023-10-16T08:18:03.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f96455d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964015cba0 0x7f96400d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f96455d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f96400b84e0 0x7f96400cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f96455d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400d1670 con 0x7f96400d0040 2023-10-16T08:18:03.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f96455d9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400d1460 con 0x7f964015cba0 2023-10-16T08:18:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f96455d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400d04b0 con 0x7f96400b84e0 2023-10-16T08:18:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964015cba0 0x7f96400d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964015cba0 0x7f96400d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40386/0 (socket says 172.21.15.32:40386) 2023-10-16T08:18:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 -- 172.21.15.32:0/2236872416 learned_addr learned my addr 172.21.15.32:0/2236872416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:03.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963f7fe640 1 --1- 172.21.15.32:0/2236872416 >> v1:172.21.15.32:6789/0 conn(0x7f96400d0040 0x7f96400d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59136/0 (socket says 172.21.15.32:59136) 2023-10-16T08:18:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f9637fff640 1 --2- 172.21.15.32:0/2236872416 >> v2:172.21.15.186:3300/0 conn(0x7f96400b84e0 0x7f96400cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 -- 172.21.15.32:0/2236872416 >> v2:172.21.15.186:3300/0 conn(0x7f96400b84e0 msgr2=0x7f96400cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 --2- 172.21.15.32:0/2236872416 >> v2:172.21.15.186:3300/0 conn(0x7f96400b84e0 0x7f96400cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 -- 172.21.15.32:0/2236872416 >> v1:172.21.15.32:6789/0 conn(0x7f96400d0040 legacy=0x7f96400d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.929+0000 7f963effd640 1 -- 172.21.15.32:0/2236872416 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400d1950 con 0x7f964015cba0 2023-10-16T08:18:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f963effd640 1 --2- 172.21.15.32:0/2236872416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964015cba0 0x7f96400d0f20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f96400cf2f0 tx=0x7f963001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6b89323e4adee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f963e7fc640 1 -- 172.21.15.32:0/2236872416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300a8050 con 0x7f964015cba0 2023-10-16T08:18:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f963e7fc640 1 -- 172.21.15.32:0/2236872416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96300ac5f0 con 0x7f964015cba0 2023-10-16T08:18:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f963e7fc640 1 -- 172.21.15.32:0/2236872416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300ac980 con 0x7f964015cba0 2023-10-16T08:18:03.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- 172.21.15.32:0/2236872416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964015cba0 msgr2=0x7f96400d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 --2- 172.21.15.32:0/2236872416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964015cba0 0x7f96400d0f20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f96400cf2f0 tx=0x7f963001c470 comp rx=0 tx=0).stop 2023-10-16T08:18:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- 172.21.15.32:0/2236872416 shutdown_connections 2023-10-16T08:18:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 --2- 172.21.15.32:0/2236872416 >> v2:172.21.15.186:3300/0 conn(0x7f96400b84e0 0x7f96400cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 --2- 172.21.15.32:0/2236872416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964015cba0 0x7f96400d0f20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- 172.21.15.32:0/2236872416 >> 172.21.15.32:0/2236872416 conn(0x7f96400719c0 msgr2=0x7f9640071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- 172.21.15.32:0/2236872416 shutdown_connections 2023-10-16T08:18:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- 172.21.15.32:0/2236872416 wait complete. 2023-10-16T08:18:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 Processor -- start 2023-10-16T08:18:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- start start 2023-10-16T08:18:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 0x7f96400c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96400d0040 0x7f9640146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964015cba0 0x7f96400c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400c5b50 con 0x7f96400b84e0 2023-10-16T08:18:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964014bda0 con 0x7f96400d0040 2023-10-16T08:18:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.933+0000 7f96455d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964014c080 con 0x7f964015cba0 2023-10-16T08:18:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 0x7f96400c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 0x7f96400c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40390/0 (socket says 172.21.15.32:40390) 2023-10-16T08:18:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 -- 172.21.15.32:0/192525685 learned_addr learned my addr 172.21.15.32:0/192525685 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963f7fe640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964015cba0 0x7f96400c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f9637fff640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96400d0040 0x7f9640146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 -- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964015cba0 msgr2=0x7f96400c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964015cba0 0x7f96400c3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 -- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96400d0040 msgr2=0x7f9640146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96400d0040 0x7f9640146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 -- 172.21.15.32:0/192525685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300094d0 con 0x7f96400b84e0 2023-10-16T08:18:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f963effd640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 0x7f96400c5320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f963001ab20 tx=0x7f9630019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300a8040 con 0x7f96400b84e0 2023-10-16T08:18:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96300b45c0 con 0x7f96400b84e0 2023-10-16T08:18:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300b4950 con 0x7f96400b84e0 2023-10-16T08:18:03.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964014c360 con 0x7f96400b84e0 2023-10-16T08:18:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96400c5860 con 0x7f96400b84e0 2023-10-16T08:18:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964004e170 con 0x7f96400b84e0 2023-10-16T08:18:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96300b4ab0 con 0x7f96400b84e0 2023-10-16T08:18:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.937+0000 7f96377fe640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96200664f0 0x7f96200689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.941+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f963011e2c0 con 0x7f96400b84e0 2023-10-16T08:18:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.941+0000 7f9637fff640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96200664f0 0x7f96200689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.941+0000 7f9637fff640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96200664f0 0x7f96200689b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f96400c70e0 tx=0x7f9628008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:03.941+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96300eb5e0 con 0x7f96400b84e0 2023-10-16T08:18:04.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f964014c810 con 0x7f96400b84e0 2023-10-16T08:18:04.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96377fe640 1 -- 172.21.15.32:0/192525685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7f96300eec00 con 0x7f96400b84e0 2023-10-16T08:18:04.107 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96200664f0 msgr2=0x7f96200689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96200664f0 0x7f96200689b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f96400c70e0 tx=0x7f9628008040 comp rx=0 tx=0).stop 2023-10-16T08:18:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 msgr2=0x7f96400c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 0x7f96400c5320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f963001ab20 tx=0x7f9630019210 comp rx=0 tx=0).stop 2023-10-16T08:18:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 shutdown_connections 2023-10-16T08:18:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96200664f0 0x7f96200689b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f964015cba0 0x7f96400c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96400d0040 0x7f9640146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 --2- 172.21.15.32:0/192525685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96400b84e0 0x7f96400c5320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 >> 172.21.15.32:0/192525685 conn(0x7f96400719c0 msgr2=0x7f964006b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 shutdown_connections 2023-10-16T08:18:04.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.101+0000 7f96455d9640 1 -- 172.21.15.32:0/192525685 wait complete. 2023-10-16T08:18:04.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 Processor -- start 2023-10-16T08:18:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 -- start start 2023-10-16T08:18:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f517415cba0 0x7f51740d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 0x7f51740cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740d1670 con 0x7f51740d0040 2023-10-16T08:18:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51740d1460 con 0x7f517415cba0 2023-10-16T08:18:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517c31d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d04b0 con 0x7f51740b84e0 2023-10-16T08:18:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517a8ba640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51740d0040 0x7f51740d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59140/0 (socket says 172.21.15.32:59140) 2023-10-16T08:18:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517a8ba640 1 -- 172.21.15.32:0/806885068 learned_addr learned my addr 172.21.15.32:0/806885068 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51798b8640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 0x7f51740cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f517a0b9640 1 --2- 172.21.15.32:0/806885068 >> v2:172.21.15.186:3300/0 conn(0x7f517415cba0 0x7f51740d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840184749 0 0) 0x7f51740d1670 con 0x7f51740d0040 2023-10-16T08:18:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5160003530 con 0x7f51740d0040 2023-10-16T08:18:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019004159 0 0) 0x7f5160003530 con 0x7f51740d0040 2023-10-16T08:18:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 msgr2=0x7f51740cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 0x7f51740cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 >> v2:172.21.15.186:3300/0 conn(0x7f517415cba0 msgr2=0x7f51740d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 --2- 172.21.15.32:0/806885068 >> v2:172.21.15.186:3300/0 conn(0x7f517415cba0 0x7f51740d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740d1950 con 0x7f51740d0040 2023-10-16T08:18:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f516c001a00 con 0x7f51740d0040 2023-10-16T08:18:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.269+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 >> v1:172.21.15.32:6789/0 conn(0x7f51740d0040 legacy=0x7f51740d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160003980 0x7f5160003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 0x7f51600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160004d10 0x7f5160005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d1460 con 0x7f5160004d10 2023-10-16T08:18:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d04b0 con 0x7f5160003980 2023-10-16T08:18:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516000b990 con 0x7f51740b84e0 2023-10-16T08:18:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160003980 0x7f5160003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a8ba640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160004d10 0x7f5160005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51798b8640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 0x7f51600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 -- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 msgr2=0x7f51600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51740b84e0 0x7f51600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 -- 172.21.15.32:0/806885068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160004d10 msgr2=0x7f5160005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160004d10 0x7f5160005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 -- 172.21.15.32:0/806885068 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160003530 con 0x7f5160003980 2023-10-16T08:18:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a8ba640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160004d10 0x7f5160005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f517a0b9640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160003980 0x7f5160003da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5170009b10 tx=0x7f517001ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=462aa6201b57b3d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700a5680 con 0x7f5160003980 2023-10-16T08:18:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51700a57e0 con 0x7f5160003980 2023-10-16T08:18:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.273+0000 7f51790b7640 1 -- 172.21.15.32:0/806885068 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700a5b90 con 0x7f5160003980 2023-10-16T08:18:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160003980 msgr2=0x7f5160003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 --2- 172.21.15.32:0/806885068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160003980 0x7f5160003da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5170009b10 tx=0x7f517001ca90 comp rx=0 tx=0).stop 2023-10-16T08:18:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517a8ba640 1 -- 172.21.15.32:0/806885068 reap_dead start 2023-10-16T08:18:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- 172.21.15.32:0/806885068 shutdown_connections 2023-10-16T08:18:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- 172.21.15.32:0/806885068 >> 172.21.15.32:0/806885068 conn(0x7f51740719c0 msgr2=0x7f5174071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- 172.21.15.32:0/806885068 shutdown_connections 2023-10-16T08:18:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- 172.21.15.32:0/806885068 wait complete. 2023-10-16T08:18:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 Processor -- start 2023-10-16T08:18:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- start start 2023-10-16T08:18:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5160003980 0x7f51740c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51740c75c0 0x7f517414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 0x7f51741581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740c8a50 con 0x7f5174157d50 2023-10-16T08:18:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174158710 con 0x7f51740c75c0 2023-10-16T08:18:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517c31d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740c8e70 con 0x7f5160003980 2023-10-16T08:18:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517a8ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 0x7f51741581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517a8ba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 0x7f51741581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40408/0 (socket says 172.21.15.32:40408) 2023-10-16T08:18:04.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517a8ba640 1 -- 172.21.15.32:0/433849229 learned_addr learned my addr 172.21.15.32:0/433849229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f51798b8640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51740c75c0 0x7f517414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.277+0000 7f517a0b9640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5160003980 0x7f51740c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517a8ba640 1 -- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5160003980 msgr2=0x7f51740c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517a8ba640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5160003980 0x7f51740c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517a8ba640 1 -- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51740c75c0 msgr2=0x7f517414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517a8ba640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51740c75c0 0x7f517414f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517a8ba640 1 -- 172.21.15.32:0/433849229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740c7dd0 con 0x7f5174157d50 2023-10-16T08:18:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517a8ba640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 0x7f51741581d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f51740d06b0 tx=0x7f516c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c0078c0 con 0x7f5174157d50 2023-10-16T08:18:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c007a40 con 0x7f5174157d50 2023-10-16T08:18:04.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c013630 con 0x7f5174157d50 2023-10-16T08:18:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51740c80b0 con 0x7f5174157d50 2023-10-16T08:18:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51740c8670 con 0x7f5174157d50 2023-10-16T08:18:04.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51740c8b90 con 0x7f5174157d50 2023-10-16T08:18:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f516c013790 con 0x7f5174157d50 2023-10-16T08:18:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f516affd640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5158066520 0x7f51580689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.281+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f516c084920 con 0x7f5174157d50 2023-10-16T08:18:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.285+0000 7f517a0b9640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5158066520 0x7f51580689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.285+0000 7f517a0b9640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5158066520 0x7f51580689e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51700af000 tx=0x7f51700046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.285+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f51740c8b90 con 0x7f5174157d50 2023-10-16T08:18:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f51740c8b90 con 0x7f5174157d50 2023-10-16T08:18:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f516affd640 1 -- 172.21.15.32:0/433849229 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7f51740c8b90 con 0x7f5174157d50 2023-10-16T08:18:04.452 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:04.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5158066520 msgr2=0x7f51580689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5158066520 0x7f51580689e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51700af000 tx=0x7f51700046d0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 msgr2=0x7f51741581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 0x7f51741581d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f51740d06b0 tx=0x7f516c00af90 comp rx=0 tx=0).stop 2023-10-16T08:18:04.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 shutdown_connections 2023-10-16T08:18:04.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5174157d50 0x7f51741581d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51740c75c0 0x7f517414f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5158066520 0x7f51580689e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 --2- 172.21.15.32:0/433849229 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5160003980 0x7f51740c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 >> 172.21.15.32:0/433849229 conn(0x7f51740719c0 msgr2=0x7f5174159990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:04.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 shutdown_connections 2023-10-16T08:18:04.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.445+0000 7f517c31d640 1 -- 172.21.15.32:0/433849229 wait complete. 2023-10-16T08:18:04.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 Processor -- start 2023-10-16T08:18:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 -- start start 2023-10-16T08:18:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff52015cc60 0x7ff5200cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff520161260 0x7ff5200ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5200cc330 con 0x7ff5200cebd0 2023-10-16T08:18:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cefe0 con 0x7ff520161260 2023-10-16T08:18:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff52769c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cb720 con 0x7ff52015cc60 2023-10-16T08:18:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff524c37640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff520161260 0x7ff5200ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff524c37640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff520161260 0x7ff5200ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40422/0 (socket says 172.21.15.32:40422) 2023-10-16T08:18:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.613+0000 7ff524c37640 1 -- 172.21.15.32:0/522609987 learned_addr learned my addr 172.21.15.32:0/522609987 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff525438640 1 --2- 172.21.15.32:0/522609987 >> v2:172.21.15.186:3300/0 conn(0x7ff52015cc60 0x7ff5200cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff524c37640 1 -- 172.21.15.32:0/522609987 >> v2:172.21.15.186:3300/0 conn(0x7ff52015cc60 msgr2=0x7ff5200cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff524c37640 1 --2- 172.21.15.32:0/522609987 >> v2:172.21.15.186:3300/0 conn(0x7ff52015cc60 0x7ff5200cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff524c37640 1 -- 172.21.15.32:0/522609987 >> v1:172.21.15.186:6789/0 conn(0x7ff5200cebd0 legacy=0x7ff5200b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff524c37640 1 -- 172.21.15.32:0/522609987 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200cba00 con 0x7ff520161260 2023-10-16T08:18:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff524c37640 1 --2- 172.21.15.32:0/522609987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff520161260 0x7ff5200ce690 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5200d0810 tx=0x7ff51001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ce07b2fd828d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff517fff640 1 -- 172.21.15.32:0/522609987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100aa430 con 0x7ff520161260 2023-10-16T08:18:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff517fff640 1 -- 172.21.15.32:0/522609987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5100aa590 con 0x7ff520161260 2023-10-16T08:18:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff517fff640 1 -- 172.21.15.32:0/522609987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100aa920 con 0x7ff520161260 2023-10-16T08:18:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff52769c640 1 -- 172.21.15.32:0/522609987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff520161260 msgr2=0x7ff5200ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.617+0000 7ff52769c640 1 --2- 172.21.15.32:0/522609987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff520161260 0x7ff5200ce690 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5200d0810 tx=0x7ff51001c4d0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- 172.21.15.32:0/522609987 shutdown_connections 2023-10-16T08:18:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 --2- 172.21.15.32:0/522609987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff520161260 0x7ff5200ce690 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 --2- 172.21.15.32:0/522609987 >> v2:172.21.15.186:3300/0 conn(0x7ff52015cc60 0x7ff5200cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- 172.21.15.32:0/522609987 >> 172.21.15.32:0/522609987 conn(0x7ff52006e570 msgr2=0x7ff52006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- 172.21.15.32:0/522609987 shutdown_connections 2023-10-16T08:18:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- 172.21.15.32:0/522609987 wait complete. 2023-10-16T08:18:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 Processor -- start 2023-10-16T08:18:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- start start 2023-10-16T08:18:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5200cebd0 0x7ff5200b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 0x7ff520153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff520161260 0x7ff5201579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200b5970 con 0x7ff52015cc60 2023-10-16T08:18:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201582b0 con 0x7ff5200cebd0 2023-10-16T08:18:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff52769c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff520158590 con 0x7ff520161260 2023-10-16T08:18:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 0x7ff520153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 0x7ff520153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40432/0 (socket says 172.21.15.32:40432) 2023-10-16T08:18:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 -- 172.21.15.32:0/114726975 learned_addr learned my addr 172.21.15.32:0/114726975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff525438640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5200cebd0 0x7ff5200b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff525c39640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff520161260 0x7ff5201579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 -- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff520161260 msgr2=0x7ff5201579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff520161260 0x7ff5201579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 -- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5200cebd0 msgr2=0x7ff5200b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5200cebd0 0x7ff5200b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.621+0000 7ff524c37640 1 -- 172.21.15.32:0/114726975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100094d0 con 0x7ff52015cc60 2023-10-16T08:18:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff524c37640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 0x7ff520153f00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff5100190e0 tx=0x7ff51001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100aa430 con 0x7ff52015cc60 2023-10-16T08:18:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5100b3800 con 0x7ff52015cc60 2023-10-16T08:18:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b3bb0 con 0x7ff52015cc60 2023-10-16T08:18:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200b4c30 con 0x7ff52015cc60 2023-10-16T08:18:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520154440 con 0x7ff52015cc60 2023-10-16T08:18:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200b4a90 con 0x7ff52015cc60 2023-10-16T08:18:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5100b3d10 con 0x7ff52015cc60 2023-10-16T08:18:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff515ffb640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff5040664a0 0x7ff504068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7ff51011e5e0 con 0x7ff52015cc60 2023-10-16T08:18:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.625+0000 7ff525438640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff5040664a0 0x7ff504068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.629+0000 7ff525438640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff5040664a0 0x7ff504068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b9d0 tx=0x7ff51c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.629+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5200b4a90 con 0x7ff52015cc60 2023-10-16T08:18:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.789+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7ff5200b4a90 con 0x7ff52015cc60 2023-10-16T08:18:04.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.789+0000 7ff515ffb640 1 -- 172.21.15.32:0/114726975 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7ff5200b4a90 con 0x7ff52015cc60 2023-10-16T08:18:04.797 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff5040664a0 msgr2=0x7ff504068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff5040664a0 0x7ff504068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b9d0 tx=0x7ff51c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 msgr2=0x7ff520153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 0x7ff520153f00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff5100190e0 tx=0x7ff51001ad10 comp rx=0 tx=0).stop 2023-10-16T08:18:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 shutdown_connections 2023-10-16T08:18:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff5040664a0 0x7ff504068960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff520161260 0x7ff5201579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff52015cc60 0x7ff520153f00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 --2- 172.21.15.32:0/114726975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5200cebd0 0x7ff5200b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 >> 172.21.15.32:0/114726975 conn(0x7ff52006e570 msgr2=0x7ff52006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 shutdown_connections 2023-10-16T08:18:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.793+0000 7ff52769c640 1 -- 172.21.15.32:0/114726975 wait complete. 2023-10-16T08:18:04.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f26a5f3e640 1 Processor -- start 2023-10-16T08:18:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f26a5f3e640 1 -- start start 2023-10-16T08:18:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f26a5f3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f26a00d15d0 0x7f26a00cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f26a5f3e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a00ce770 con 0x7f26a00d3010 2023-10-16T08:18:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f26a5f3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a00d0600 con 0x7f26a015cb90 2023-10-16T08:18:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f26a5f3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a00c6450 con 0x7f26a00d15d0 2023-10-16T08:18:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f269ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26a00d3010 0x7f26a00c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59146/0 (socket says 172.21.15.32:59146) 2023-10-16T08:18:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.957+0000 7f269ffff640 1 -- 172.21.15.32:0/652620003 learned_addr learned my addr 172.21.15.32:0/652620003 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269effd640 1 --2- 172.21.15.32:0/652620003 >> v2:172.21.15.186:3300/0 conn(0x7f26a00d15d0 0x7f26a00cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269e7fc640 1 -- 172.21.15.32:0/652620003 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151828801 0 0) 0x7f26a00ce770 con 0x7f26a00d3010 2023-10-16T08:18:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269e7fc640 1 -- 172.21.15.32:0/652620003 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2688003530 con 0x7f26a00d3010 2023-10-16T08:18:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269effd640 1 -- 172.21.15.32:0/652620003 >> v1:172.21.15.186:6789/0 conn(0x7f26a015cb90 legacy=0x7f26a00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269effd640 1 -- 172.21.15.32:0/652620003 >> v1:172.21.15.32:6789/0 conn(0x7f26a00d3010 legacy=0x7f26a00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269effd640 1 -- 172.21.15.32:0/652620003 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00d08a0 con 0x7f26a00d15d0 2023-10-16T08:18:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269effd640 1 --2- 172.21.15.32:0/652620003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d15d0 0x7f26a00cfb10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26940097c0 tx=0x7f269401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31de6eb3bcf2d27c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269e7fc640 1 -- 172.21.15.32:0/652620003 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940ac620 con 0x7f26a00d15d0 2023-10-16T08:18:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269e7fc640 1 -- 172.21.15.32:0/652620003 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940ac780 con 0x7f26a00d15d0 2023-10-16T08:18:04.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.961+0000 7f269e7fc640 1 -- 172.21.15.32:0/652620003 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940acb10 con 0x7f26a00d15d0 2023-10-16T08:18:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- 172.21.15.32:0/652620003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d15d0 msgr2=0x7f26a00cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/652620003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d15d0 0x7f26a00cfb10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26940097c0 tx=0x7f269401c3c0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- 172.21.15.32:0/652620003 shutdown_connections 2023-10-16T08:18:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/652620003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d15d0 0x7f26a00cfb10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- 172.21.15.32:0/652620003 >> 172.21.15.32:0/652620003 conn(0x7f26a009bc50 msgr2=0x7f26a00d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- 172.21.15.32:0/652620003 shutdown_connections 2023-10-16T08:18:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- 172.21.15.32:0/652620003 wait complete. 2023-10-16T08:18:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 Processor -- start 2023-10-16T08:18:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- start start 2023-10-16T08:18:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26a00d15d0 0x7f26a00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d3010 0x7f26a0153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 0x7f26a00cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0153e60 con 0x7f26a015cb90 2023-10-16T08:18:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0154140 con 0x7f26a00d3010 2023-10-16T08:18:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f26a5f3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0154420 con 0x7f26a00d15d0 2023-10-16T08:18:04.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f269ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 0x7f26a00cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f269ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 0x7f26a00cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40440/0 (socket says 172.21.15.32:40440) 2023-10-16T08:18:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f269ffff640 1 -- 172.21.15.32:0/127268559 learned_addr learned my addr 172.21.15.32:0/127268559 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f269f7fe640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26a00d15d0 0x7f26a00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f269effd640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d3010 0x7f26a0153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.965+0000 7f269ffff640 1 -- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26a00d15d0 msgr2=0x7f26a00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f269ffff640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26a00d15d0 0x7f26a00cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f269ffff640 1 -- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d3010 msgr2=0x7f26a0153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f269ffff640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d3010 0x7f26a0153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f269ffff640 1 -- 172.21.15.32:0/127268559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694009420 con 0x7f26a015cb90 2023-10-16T08:18:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f269ffff640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 0x7f26a00cdc50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f269800cb40 tx=0x7f269800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269800aca0 con 0x7f26a015cb90 2023-10-16T08:18:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269800ae00 con 0x7f26a015cb90 2023-10-16T08:18:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698012620 con 0x7f26a015cb90 2023-10-16T08:18:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a0154700 con 0x7f26a015cb90 2023-10-16T08:18:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a00cc020 con 0x7f26a015cb90 2023-10-16T08:18:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a0074010 con 0x7f26a015cb90 2023-10-16T08:18:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2698007070 con 0x7f26a015cb90 2023-10-16T08:18:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f2687fff640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f268c0664a0 0x7f268c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f26980841f0 con 0x7f26a015cb90 2023-10-16T08:18:04.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.969+0000 7f269f7fe640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f268c0664a0 0x7f268c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.973+0000 7f269f7fe640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f268c0664a0 0x7f268c068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2690000fc0 tx=0x7f2690009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:04.973+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2698051380 con 0x7f26a015cb90 2023-10-16T08:18:05.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.145+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26a0154bb0 con 0x7f26a015cb90 2023-10-16T08:18:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.145+0000 7f2687fff640 1 -- 172.21.15.32:0/127268559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26980549a0 con 0x7f26a015cb90 2023-10-16T08:18:05.153 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f268c0664a0 msgr2=0x7f268c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f268c0664a0 0x7f268c068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2690000fc0 tx=0x7f2690009040 comp rx=0 tx=0).stop 2023-10-16T08:18:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 msgr2=0x7f26a00cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 0x7f26a00cdc50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f269800cb40 tx=0x7f269800a060 comp rx=0 tx=0).stop 2023-10-16T08:18:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 shutdown_connections 2023-10-16T08:18:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a015cb90 0x7f26a00cdc50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26a00d3010 0x7f26a0153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f268c0664a0 0x7f268c068960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 --2- 172.21.15.32:0/127268559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f26a00d15d0 0x7f26a00cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 >> 172.21.15.32:0/127268559 conn(0x7f26a009bc50 msgr2=0x7f26a006bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 shutdown_connections 2023-10-16T08:18:05.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.149+0000 7f26a5f3e640 1 -- 172.21.15.32:0/127268559 wait complete. 2023-10-16T08:18:05.174 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:05.174 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:05.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:05.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 Processor -- start 2023-10-16T08:18:05.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 -- start start 2023-10-16T08:18:05.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe3415cb90 0x7fbe340cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbe340d15d0 0x7fbe340c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe340ce770 con 0x7fbe340d3010 2023-10-16T08:18:05.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe340d0600 con 0x7fbe3415cb90 2023-10-16T08:18:05.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe3afd9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe340c6450 con 0x7fbe340d15d0 2023-10-16T08:18:05.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe38d75640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe3415cb90 0x7fbe340cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe33fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbe340d3010 0x7fbe340d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59158/0 (socket says 172.21.15.32:59158) 2023-10-16T08:18:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe33fff640 1 -- 172.21.15.32:0/2761170069 learned_addr learned my addr 172.21.15.32:0/2761170069 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe39576640 1 --2- 172.21.15.32:0/2761170069 >> v2:172.21.15.186:3300/0 conn(0x7fbe340d15d0 0x7fbe340c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe38d75640 1 -- 172.21.15.32:0/2761170069 >> v2:172.21.15.186:3300/0 conn(0x7fbe340d15d0 msgr2=0x7fbe340c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe38d75640 1 --2- 172.21.15.32:0/2761170069 >> v2:172.21.15.186:3300/0 conn(0x7fbe340d15d0 0x7fbe340c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe38d75640 1 -- 172.21.15.32:0/2761170069 >> v1:172.21.15.32:6789/0 conn(0x7fbe340d3010 legacy=0x7fbe340d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe38d75640 1 -- 172.21.15.32:0/2761170069 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340d08a0 con 0x7fbe3415cb90 2023-10-16T08:18:05.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe337fe640 1 -- 172.21.15.32:0/2761170069 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62710879 0 0) 0x7fbe340ce770 con 0x7fbe340d3010 2023-10-16T08:18:05.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.365+0000 7fbe39576640 1 --2- 172.21.15.32:0/2761170069 >> v2:172.21.15.186:3300/0 conn(0x7fbe340d15d0 0x7fbe340c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe38d75640 1 --2- 172.21.15.32:0/2761170069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340cfb10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe340ccee0 tx=0x7fbe2801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566aeeec332d751b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe337fe640 1 -- 172.21.15.32:0/2761170069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280ad680 con 0x7fbe3415cb90 2023-10-16T08:18:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe337fe640 1 -- 172.21.15.32:0/2761170069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe280ad7e0 con 0x7fbe3415cb90 2023-10-16T08:18:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe337fe640 1 -- 172.21.15.32:0/2761170069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280adb70 con 0x7fbe3415cb90 2023-10-16T08:18:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2761170069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 msgr2=0x7fbe340cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2761170069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340cfb10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe340ccee0 tx=0x7fbe2801c380 comp rx=0 tx=0).stop 2023-10-16T08:18:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2761170069 shutdown_connections 2023-10-16T08:18:05.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2761170069 >> v2:172.21.15.186:3300/0 conn(0x7fbe340d15d0 0x7fbe340c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2761170069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340cfb10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2761170069 >> 172.21.15.32:0/2761170069 conn(0x7fbe3409bc50 msgr2=0x7fbe340d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:05.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2761170069 shutdown_connections 2023-10-16T08:18:05.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.369+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2761170069 wait complete. 2023-10-16T08:18:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 Processor -- start 2023-10-16T08:18:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 -- start start 2023-10-16T08:18:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe340d15d0 0x7fbe34156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe340d3010 0x7fbe340b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b69f0 con 0x7fbe3415cb90 2023-10-16T08:18:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b6cd0 con 0x7fbe340d3010 2023-10-16T08:18:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b6fb0 con 0x7fbe340d15d0 2023-10-16T08:18:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40468/0 (socket says 172.21.15.32:40468) 2023-10-16T08:18:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 -- 172.21.15.32:0/2336455685 learned_addr learned my addr 172.21.15.32:0/2336455685 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe38d75640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe340d15d0 0x7fbe34156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe33fff640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe340d3010 0x7fbe340b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 -- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe340d15d0 msgr2=0x7fbe34156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe340d15d0 0x7fbe34156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 -- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe340d3010 msgr2=0x7fbe340b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe340d3010 0x7fbe340b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 -- 172.21.15.32:0/2336455685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280093b0 con 0x7fbe3415cb90 2023-10-16T08:18:05.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe39576640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340c99d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00cef0 tx=0x7fbe2c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:05.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00d840 con 0x7fbe3415cb90 2023-10-16T08:18:05.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00d9c0 con 0x7fbe3415cb90 2023-10-16T08:18:05.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe340b7290 con 0x7fbe3415cb90 2023-10-16T08:18:05.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00dd70 con 0x7fbe3415cb90 2023-10-16T08:18:05.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.373+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe34158330 con 0x7fbe3415cb90 2023-10-16T08:18:05.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.377+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe34074010 con 0x7fbe3415cb90 2023-10-16T08:18:05.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.377+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe2c01e3b0 con 0x7fbe3415cb90 2023-10-16T08:18:05.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.377+0000 7fbe317fa640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe180664a0 0x7fbe18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.377+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fbe2c084cc0 con 0x7fbe3415cb90 2023-10-16T08:18:05.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.377+0000 7fbe38d75640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe180664a0 0x7fbe18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.377+0000 7fbe38d75640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe180664a0 0x7fbe18068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbe280af000 tx=0x7fbe280a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:05.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.381+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe2c08b080 con 0x7fbe3415cb90 2023-10-16T08:18:05.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.537+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe340ce810 con 0x7fbe3415cb90 2023-10-16T08:18:05.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe317fa640 1 -- 172.21.15.32:0/2336455685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7fbe2c01a020 con 0x7fbe3415cb90 2023-10-16T08:18:05.547 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:05.547 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe180664a0 msgr2=0x7fbe18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe180664a0 0x7fbe18068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbe280af000 tx=0x7fbe280a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 msgr2=0x7fbe340c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340c99d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00cef0 tx=0x7fbe2c00a710 comp rx=0 tx=0).stop 2023-10-16T08:18:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 shutdown_connections 2023-10-16T08:18:05.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe3415cb90 0x7fbe340c99d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe340d3010 0x7fbe340b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbe180664a0 0x7fbe18068960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 --2- 172.21.15.32:0/2336455685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbe340d15d0 0x7fbe34156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 >> 172.21.15.32:0/2336455685 conn(0x7fbe3409bc50 msgr2=0x7fbe3406ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:05.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 shutdown_connections 2023-10-16T08:18:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.541+0000 7fbe3afd9640 1 -- 172.21.15.32:0/2336455685 wait complete. 2023-10-16T08:18:05.568 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:05.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 Processor -- start 2023-10-16T08:18:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 -- start start 2023-10-16T08:18:05.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f991c069e90 0x7f991c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06a270 0x7f991c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c15ae30 con 0x7f991c0702f0 2023-10-16T08:18:05.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0cb090 con 0x7f991c069e90 2023-10-16T08:18:05.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9924217640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0ca510 con 0x7f991c06a270 2023-10-16T08:18:05.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f9921fb3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f991c069e90 0x7f991c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.709+0000 7f99217b2640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06a270 0x7f991c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f991c069e90 0x7f991c06d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:37442/0 (socket says 172.21.15.32:37442) 2023-10-16T08:18:05.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 -- 172.21.15.32:0/174078516 learned_addr learned my addr 172.21.15.32:0/174078516 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:05.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 -- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06a270 msgr2=0x7f991c0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:05.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 --2- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06a270 0x7f991c0c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 -- 172.21.15.32:0/174078516 >> v1:172.21.15.186:6789/0 conn(0x7f991c0702f0 legacy=0x7f991c070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 -- 172.21.15.32:0/174078516 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0ca7f0 con 0x7f991c069e90 2023-10-16T08:18:05.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9921fb3640 1 --2- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c069e90 0x7f991c06d010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9918009870 tx=0x7f991801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d19294ac970684 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:05.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9920fb1640 1 -- 172.21.15.32:0/174078516 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180aa670 con 0x7f991c069e90 2023-10-16T08:18:05.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9920fb1640 1 -- 172.21.15.32:0/174078516 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99180aa7d0 con 0x7f991c069e90 2023-10-16T08:18:05.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.713+0000 7f9920fb1640 1 -- 172.21.15.32:0/174078516 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180aab60 con 0x7f991c069e90 2023-10-16T08:18:05.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c069e90 msgr2=0x7f991c06d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 --2- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c069e90 0x7f991c06d010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9918009870 tx=0x7f991801c510 comp rx=0 tx=0).stop 2023-10-16T08:18:05.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- 172.21.15.32:0/174078516 shutdown_connections 2023-10-16T08:18:05.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 --2- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c06a270 0x7f991c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 --2- 172.21.15.32:0/174078516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c069e90 0x7f991c06d010 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- 172.21.15.32:0/174078516 >> 172.21.15.32:0/174078516 conn(0x7f991c0b0da0 msgr2=0x7f991c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:05.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- 172.21.15.32:0/174078516 shutdown_connections 2023-10-16T08:18:05.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- 172.21.15.32:0/174078516 wait complete. 2023-10-16T08:18:05.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 Processor -- start 2023-10-16T08:18:05.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- start start 2023-10-16T08:18:05.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c069e90 0x7f991c0ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 0x7f991c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c0702f0 0x7f991c15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0ac060 con 0x7f991c06a270 2023-10-16T08:18:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c15d3b0 con 0x7f991c0702f0 2023-10-16T08:18:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9924217640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c15d0d0 con 0x7f991c069e90 2023-10-16T08:18:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 0x7f991c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 0x7f991c0cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40472/0 (socket says 172.21.15.32:40472) 2023-10-16T08:18:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 -- 172.21.15.32:0/792347050 learned_addr learned my addr 172.21.15.32:0/792347050 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9921fb3640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c069e90 0x7f991c0ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99227b4640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c0702f0 0x7f991c15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 -- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c069e90 msgr2=0x7f991c0ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c069e90 0x7f991c0ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 -- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c0702f0 msgr2=0x7f991c15c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c0702f0 0x7f991c15c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99217b2640 1 -- 172.21.15.32:0/792347050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180094d0 con 0x7f991c06a270 2023-10-16T08:18:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f9921fb3640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c069e90 0x7f991c0ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.717+0000 7f99227b4640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c0702f0 0x7f991c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99217b2640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 0x7f991c0cb6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f990c00e930 tx=0x7f990c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c009690 con 0x7f991c06a270 2023-10-16T08:18:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00ca00 con 0x7f991c06a270 2023-10-16T08:18:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00cd70 con 0x7f991c06a270 2023-10-16T08:18:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c0ab9a0 con 0x7f991c06a270 2023-10-16T08:18:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c0cbc30 con 0x7f991c06a270 2023-10-16T08:18:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c04e170 con 0x7f991c06a270 2023-10-16T08:18:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f990c013470 con 0x7f991c06a270 2023-10-16T08:18:05.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99127fc640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f99000664d0 0x7f9900068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:05.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f990c084350 con 0x7f991c06a270 2023-10-16T08:18:05.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.721+0000 7f9921fb3640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f99000664d0 0x7f9900068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:05.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.725+0000 7f9921fb3640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f99000664d0 0x7f9900068990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f99180ae000 tx=0x7f99180a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:05.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.725+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f990c051590 con 0x7f991c06a270 2023-10-16T08:18:05.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.885+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f991c0c2090 con 0x7f991c06a270 2023-10-16T08:18:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.885+0000 7f99127fc640 1 -- 172.21.15.32:0/792347050 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f990c054bb0 con 0x7f991c06a270 2023-10-16T08:18:05.893 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f99000664d0 msgr2=0x7f9900068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f99000664d0 0x7f9900068990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f99180ae000 tx=0x7f99180a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 msgr2=0x7f991c0cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 0x7f991c0cb6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f990c00e930 tx=0x7f990c00edf0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 shutdown_connections 2023-10-16T08:18:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f991c0702f0 0x7f991c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f991c06a270 0x7f991c0cb6f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f99000664d0 0x7f9900068990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 --2- 172.21.15.32:0/792347050 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f991c069e90 0x7f991c0ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 >> 172.21.15.32:0/792347050 conn(0x7f991c0b0da0 msgr2=0x7f991c0af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 shutdown_connections 2023-10-16T08:18:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:05.889+0000 7f9924217640 1 -- 172.21.15.32:0/792347050 wait complete. 2023-10-16T08:18:05.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 Processor -- start 2023-10-16T08:18:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 -- start start 2023-10-16T08:18:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a406d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a406a270 0x7f27a40c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a415ae30 con 0x7f27a40702f0 2023-10-16T08:18:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a40cb090 con 0x7f27a406a270 2023-10-16T08:18:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27ab279640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a40ca510 con 0x7f27a4069e90 2023-10-16T08:18:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a8814640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a406a270 0x7f27a40c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a9816640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f27a40702f0 0x7f27a4070780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59162/0 (socket says 172.21.15.32:59162) 2023-10-16T08:18:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a8814640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a406a270 0x7f27a40c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40486/0 (socket says 172.21.15.32:40486) 2023-10-16T08:18:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a9816640 1 -- 172.21.15.32:0/1074113011 learned_addr learned my addr 172.21.15.32:0/1074113011 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a9015640 1 --2- 172.21.15.32:0/1074113011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a8814640 1 -- 172.21.15.32:0/1074113011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 msgr2=0x7f27a406d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a8814640 1 --2- 172.21.15.32:0/1074113011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a406d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a8814640 1 -- 172.21.15.32:0/1074113011 >> v1:172.21.15.32:6789/0 conn(0x7f27a40702f0 legacy=0x7f27a4070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f27a8814640 1 -- 172.21.15.32:0/1074113011 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40ca7f0 con 0x7f27a406a270 2023-10-16T08:18:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.053+0000 7f279bfff640 1 -- 172.21.15.32:0/1074113011 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267308656 0 0) 0x7f27a415ae30 con 0x7f27a40702f0 2023-10-16T08:18:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27a8814640 1 --2- 172.21.15.32:0/1074113011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a406a270 0x7f27a40c9ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f27940097c0 tx=0x7f279401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a751b8e02f3cc991 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f279bfff640 1 -- 172.21.15.32:0/1074113011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940ab650 con 0x7f27a406a270 2023-10-16T08:18:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f279bfff640 1 -- 172.21.15.32:0/1074113011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27940ab7b0 con 0x7f27a406a270 2023-10-16T08:18:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f279bfff640 1 -- 172.21.15.32:0/1074113011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940abb40 con 0x7f27a406a270 2023-10-16T08:18:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 -- 172.21.15.32:0/1074113011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a406a270 msgr2=0x7f27a40c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 --2- 172.21.15.32:0/1074113011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a406a270 0x7f27a40c9ce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f27940097c0 tx=0x7f279401c450 comp rx=0 tx=0).stop 2023-10-16T08:18:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 -- 172.21.15.32:0/1074113011 shutdown_connections 2023-10-16T08:18:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 --2- 172.21.15.32:0/1074113011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a406a270 0x7f27a40c9ce0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 --2- 172.21.15.32:0/1074113011 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a406d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 -- 172.21.15.32:0/1074113011 >> 172.21.15.32:0/1074113011 conn(0x7f27a40b0da0 msgr2=0x7f27a40ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 -- 172.21.15.32:0/1074113011 shutdown_connections 2023-10-16T08:18:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.057+0000 7f27ab279640 1 -- 172.21.15.32:0/1074113011 wait complete. 2023-10-16T08:18:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 Processor -- start 2023-10-16T08:18:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 -- start start 2023-10-16T08:18:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a4149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27a406a270 0x7f27a40c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 0x7f27a415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a4157a10 con 0x7f27a40702f0 2023-10-16T08:18:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a415b140 con 0x7f27a406a270 2023-10-16T08:18:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a415b420 con 0x7f27a4069e90 2023-10-16T08:18:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 0x7f27a415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 0x7f27a415bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40488/0 (socket says 172.21.15.32:40488) 2023-10-16T08:18:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 -- 172.21.15.32:0/513939281 learned_addr learned my addr 172.21.15.32:0/513939281 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9015640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a4149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a8814640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27a406a270 0x7f27a40c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 -- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 msgr2=0x7f27a4149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a4149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 -- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27a406a270 msgr2=0x7f27a40c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27a406a270 0x7f27a40c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 -- 172.21.15.32:0/513939281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794009420 con 0x7f27a40702f0 2023-10-16T08:18:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9015640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a4149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:06.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a8814640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27a406a270 0x7f27a40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27a9816640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 0x7f27a415bf30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f279c00c700 tx=0x7f279c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00ce10 con 0x7f27a40702f0 2023-10-16T08:18:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c00a640 con 0x7f27a40702f0 2023-10-16T08:18:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00aa70 con 0x7f27a40702f0 2023-10-16T08:18:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.061+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a415b700 con 0x7f27a40702f0 2023-10-16T08:18:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.065+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a4149860 con 0x7f27a40702f0 2023-10-16T08:18:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.065+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a404e170 con 0x7f27a40702f0 2023-10-16T08:18:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.065+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f279c00abd0 con 0x7f27a40702f0 2023-10-16T08:18:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.065+0000 7f2799ffb640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27880664d0 0x7f2788068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.065+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f279c083480 con 0x7f27a40702f0 2023-10-16T08:18:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.065+0000 7f27a9015640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27880664d0 0x7f2788068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.069+0000 7f27a9015640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27880664d0 0x7f2788068990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009840 tx=0x7f27a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.069+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f279c0506a0 con 0x7f27a40702f0 2023-10-16T08:18:06.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.225+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27a414a140 con 0x7f27a40702f0 2023-10-16T08:18:06.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.225+0000 7f2799ffb640 1 -- 172.21.15.32:0/513939281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7f279c053cc0 con 0x7f27a40702f0 2023-10-16T08:18:06.233 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27880664d0 msgr2=0x7f2788068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27880664d0 0x7f2788068990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009840 tx=0x7f27a0008040 comp rx=0 tx=0).stop 2023-10-16T08:18:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 msgr2=0x7f27a415bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 0x7f27a415bf30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f279c00c700 tx=0x7f279c00cbc0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 shutdown_connections 2023-10-16T08:18:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a40702f0 0x7f27a415bf30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27a406a270 0x7f27a40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f27880664d0 0x7f2788068990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 --2- 172.21.15.32:0/513939281 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f27a4069e90 0x7f27a4149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 >> 172.21.15.32:0/513939281 conn(0x7f27a40b0da0 msgr2=0x7f27a40afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 shutdown_connections 2023-10-16T08:18:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.229+0000 7f27ab279640 1 -- 172.21.15.32:0/513939281 wait complete. 2023-10-16T08:18:06.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:06.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f2296f3e640 1 Processor -- start 2023-10-16T08:18:06.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f2296f3e640 1 -- start start 2023-10-16T08:18:06.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f2296f3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f229006df40 0x7f22900702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f2296f3e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900c9600 con 0x7f2290070830 2023-10-16T08:18:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f2296f3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900c98e0 con 0x7f22901590d0 2023-10-16T08:18:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f2296f3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290070ca0 con 0x7f229006df40 2023-10-16T08:18:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f22954db640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2290070830 0x7f22900cc350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59166/0 (socket says 172.21.15.32:59166) 2023-10-16T08:18:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f22954db640 1 -- 172.21.15.32:0/1109476764 learned_addr learned my addr 172.21.15.32:0/1109476764 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228ffff640 1 --2- 172.21.15.32:0/1109476764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f229006df40 0x7f22900702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228f7fe640 1 -- 172.21.15.32:0/1109476764 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533564681 0 0) 0x7f22900c9600 con 0x7f2290070830 2023-10-16T08:18:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228f7fe640 1 -- 172.21.15.32:0/1109476764 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2280005740 con 0x7f2290070830 2023-10-16T08:18:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228ffff640 1 -- 172.21.15.32:0/1109476764 >> v1:172.21.15.186:6789/0 conn(0x7f22901590d0 legacy=0x7f22900c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228f7fe640 1 -- 172.21.15.32:0/1109476764 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034468683 0 0) 0x7f22900c98e0 con 0x7f22901590d0 2023-10-16T08:18:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228ffff640 1 -- 172.21.15.32:0/1109476764 >> v1:172.21.15.32:6789/0 conn(0x7f2290070830 legacy=0x7f22900cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.393+0000 7f228ffff640 1 -- 172.21.15.32:0/1109476764 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900c9bc0 con 0x7f229006df40 2023-10-16T08:18:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f228ffff640 1 --2- 172.21.15.32:0/1109476764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f229006df40 0x7f22900702f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f227c0097f0 tx=0x7f227c01c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14c76c9e0e67772 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f228f7fe640 1 -- 172.21.15.32:0/1109476764 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288001a00 con 0x7f229006df40 2023-10-16T08:18:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f228f7fe640 1 -- 172.21.15.32:0/1109476764 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c0a7050 con 0x7f229006df40 2023-10-16T08:18:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f228f7fe640 1 -- 172.21.15.32:0/1109476764 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0ab620 con 0x7f229006df40 2023-10-16T08:18:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 -- 172.21.15.32:0/1109476764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f229006df40 msgr2=0x7f22900702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 --2- 172.21.15.32:0/1109476764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f229006df40 0x7f22900702f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f227c0097f0 tx=0x7f227c01c300 comp rx=0 tx=0).stop 2023-10-16T08:18:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 -- 172.21.15.32:0/1109476764 shutdown_connections 2023-10-16T08:18:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 --2- 172.21.15.32:0/1109476764 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f229006df40 0x7f22900702f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 -- 172.21.15.32:0/1109476764 >> 172.21.15.32:0/1109476764 conn(0x7f22900b0da0 msgr2=0x7f22900ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 -- 172.21.15.32:0/1109476764 shutdown_connections 2023-10-16T08:18:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.397+0000 7f2296f3e640 1 -- 172.21.15.32:0/1109476764 wait complete. 2023-10-16T08:18:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 Processor -- start 2023-10-16T08:18:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 -- start start 2023-10-16T08:18:06.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 0x7f2290158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2290070830 0x7f2290149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22901590d0 0x7f22900b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901566d0 con 0x7f229006df40 2023-10-16T08:18:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901496b0 con 0x7f22901590d0 2023-10-16T08:18:06.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900b1960 con 0x7f2290070830 2023-10-16T08:18:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 0x7f2290158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 0x7f2290158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40492/0 (socket says 172.21.15.32:40492) 2023-10-16T08:18:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 -- 172.21.15.32:0/2473145964 learned_addr learned my addr 172.21.15.32:0/2473145964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f22954db640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22901590d0 0x7f22900b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f228ffff640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2290070830 0x7f2290149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 -- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2290070830 msgr2=0x7f2290149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2290070830 0x7f2290149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 -- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22901590d0 msgr2=0x7f22900b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22901590d0 0x7f22900b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 -- 172.21.15.32:0/2473145964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009420 con 0x7f229006df40 2023-10-16T08:18:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f22954db640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22901590d0 0x7f22900b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f228ffff640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2290070830 0x7f2290149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2294cda640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 0x7f2290158540 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f228400c500 tx=0x7f228400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284003600 con 0x7f229006df40 2023-10-16T08:18:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2284003760 con 0x7f229006df40 2023-10-16T08:18:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900ca820 con 0x7f229006df40 2023-10-16T08:18:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284003b10 con 0x7f229006df40 2023-10-16T08:18:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.401+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290156c00 con 0x7f229006df40 2023-10-16T08:18:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.405+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229004e170 con 0x7f229006df40 2023-10-16T08:18:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.405+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2284003c70 con 0x7f229006df40 2023-10-16T08:18:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.405+0000 7f228d7fa640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22740664d0 0x7f2274068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.405+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f2284083790 con 0x7f229006df40 2023-10-16T08:18:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.405+0000 7f228ffff640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22740664d0 0x7f2274068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.409+0000 7f228ffff640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22740664d0 0x7f2274068990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f227c0ad000 tx=0x7f227c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.409+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f22840509d0 con 0x7f229006df40 2023-10-16T08:18:06.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.565+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f2290150230 con 0x7f229006df40 2023-10-16T08:18:06.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.565+0000 7f228d7fa640 1 -- 172.21.15.32:0/2473145964 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7f22840170b0 con 0x7f229006df40 2023-10-16T08:18:06.572 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22740664d0 msgr2=0x7f2274068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22740664d0 0x7f2274068990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f227c0ad000 tx=0x7f227c0a6000 comp rx=0 tx=0).stop 2023-10-16T08:18:06.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 msgr2=0x7f2290158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 0x7f2290158540 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f228400c500 tx=0x7f228400c9c0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 shutdown_connections 2023-10-16T08:18:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22901590d0 0x7f22900b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22740664d0 0x7f2274068990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2290070830 0x7f2290149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 --2- 172.21.15.32:0/2473145964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f229006df40 0x7f2290158540 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 >> 172.21.15.32:0/2473145964 conn(0x7f22900b0da0 msgr2=0x7f2290150750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:06.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 shutdown_connections 2023-10-16T08:18:06.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.569+0000 7f2296f3e640 1 -- 172.21.15.32:0/2473145964 wait complete. 2023-10-16T08:18:06.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 Processor -- start 2023-10-16T08:18:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 -- start start 2023-10-16T08:18:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f46e80d0e90 0x7f46e80d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 0x7f46e80cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80d3790 con 0x7f46e80b18e0 2023-10-16T08:18:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80cfa20 con 0x7f46e80d0e90 2023-10-16T08:18:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.733+0000 7f46ee032640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80cfd00 con 0x7f46e80cd910 2023-10-16T08:18:06.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 0x7f46e80cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 0x7f46e80cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:41132/0 (socket says 172.21.15.32:41132) 2023-10-16T08:18:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 -- 172.21.15.32:0/1940468750 learned_addr learned my addr 172.21.15.32:0/1940468750 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e77fe640 1 --2- 172.21.15.32:0/1940468750 >> v2:172.21.15.186:3300/0 conn(0x7f46e80d0e90 0x7f46e80d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 -- 172.21.15.32:0/1940468750 >> v2:172.21.15.186:3300/0 conn(0x7f46e80d0e90 msgr2=0x7f46e80d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 --2- 172.21.15.32:0/1940468750 >> v2:172.21.15.186:3300/0 conn(0x7f46e80d0e90 0x7f46e80d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 -- 172.21.15.32:0/1940468750 >> v1:172.21.15.186:6789/0 conn(0x7f46e80b18e0 legacy=0x7f46e80d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 -- 172.21.15.32:0/1940468750 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80cffe0 con 0x7f46e80cd910 2023-10-16T08:18:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e7fff640 1 --2- 172.21.15.32:0/1940468750 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 0x7f46e80cddc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009e50 tx=0x7f46e001ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1d381b68290d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e67fc640 1 -- 172.21.15.32:0/1940468750 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00a56e0 con 0x7f46e80cd910 2023-10-16T08:18:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e67fc640 1 -- 172.21.15.32:0/1940468750 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e00a5840 con 0x7f46e80cd910 2023-10-16T08:18:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.737+0000 7f46e67fc640 1 -- 172.21.15.32:0/1940468750 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00a5bd0 con 0x7f46e80cd910 2023-10-16T08:18:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- 172.21.15.32:0/1940468750 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 msgr2=0x7f46e80cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 --2- 172.21.15.32:0/1940468750 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 0x7f46e80cddc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009e50 tx=0x7f46e001ca10 comp rx=0 tx=0).stop 2023-10-16T08:18:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- 172.21.15.32:0/1940468750 shutdown_connections 2023-10-16T08:18:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 --2- 172.21.15.32:0/1940468750 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e80cd910 0x7f46e80cddc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 --2- 172.21.15.32:0/1940468750 >> v2:172.21.15.186:3300/0 conn(0x7f46e80d0e90 0x7f46e80d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- 172.21.15.32:0/1940468750 >> 172.21.15.32:0/1940468750 conn(0x7f46e80d28b0 msgr2=0x7f46e80d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- 172.21.15.32:0/1940468750 shutdown_connections 2023-10-16T08:18:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- 172.21.15.32:0/1940468750 wait complete. 2023-10-16T08:18:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 Processor -- start 2023-10-16T08:18:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- start start 2023-10-16T08:18:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 0x7f46e8154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e80d0e90 0x7f46e8151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e8153da0 0x7f46e80b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8151b40 con 0x7f46e80cd910 2023-10-16T08:18:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e815a360 con 0x7f46e80d0e90 2023-10-16T08:18:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46ee032640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8153910 con 0x7f46e8153da0 2023-10-16T08:18:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 0x7f46e8154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 0x7f46e8154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40506/0 (socket says 172.21.15.32:40506) 2023-10-16T08:18:06.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46e77fe640 1 -- 172.21.15.32:0/1163791704 learned_addr learned my addr 172.21.15.32:0/1163791704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:06.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46e6ffd640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e80d0e90 0x7f46e8151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.741+0000 7f46e7fff640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e8153da0 0x7f46e80b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e77fe640 1 -- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e8153da0 msgr2=0x7f46e80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e77fe640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e8153da0 0x7f46e80b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e77fe640 1 -- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e80d0e90 msgr2=0x7f46e8151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e77fe640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e80d0e90 0x7f46e8151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e77fe640 1 -- 172.21.15.32:0/1163791704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009b30 con 0x7f46e80cd910 2023-10-16T08:18:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e77fe640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 0x7f46e8154210 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f46d800e960 tx=0x7f46d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8009690 con 0x7f46e80cd910 2023-10-16T08:18:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d800ca00 con 0x7f46e80cd910 2023-10-16T08:18:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e8152d70 con 0x7f46e80cd910 2023-10-16T08:18:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800cd70 con 0x7f46e80cd910 2023-10-16T08:18:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e8152fe0 con 0x7f46e80cd910 2023-10-16T08:18:06.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e8073fa0 con 0x7f46e80cd910 2023-10-16T08:18:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46d8013470 con 0x7f46e80cd910 2023-10-16T08:18:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46cffff640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46d40664a0 0x7f46d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f46d8084390 con 0x7f46e80cd910 2023-10-16T08:18:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.745+0000 7f46e6ffd640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46d40664a0 0x7f46d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.749+0000 7f46e6ffd640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46d40664a0 0x7f46d4068960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f46e81f87a0 tx=0x7f46dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.749+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46d8051530 con 0x7f46e80cd910 2023-10-16T08:18:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.909+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f46e80d0c70 con 0x7f46e80cd910 2023-10-16T08:18:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.909+0000 7f46cffff640 1 -- 172.21.15.32:0/1163791704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f46d8054b50 con 0x7f46e80cd910 2023-10-16T08:18:06.917 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:06.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46d40664a0 msgr2=0x7f46d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46d40664a0 0x7f46d4068960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f46e81f87a0 tx=0x7f46dc009040 comp rx=0 tx=0).stop 2023-10-16T08:18:06.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 msgr2=0x7f46e8154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:06.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 0x7f46e8154210 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f46d800e960 tx=0x7f46d800ee20 comp rx=0 tx=0).stop 2023-10-16T08:18:06.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 shutdown_connections 2023-10-16T08:18:06.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f46d40664a0 0x7f46d4068960 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f46e8153da0 0x7f46e80b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46e80d0e90 0x7f46e8151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 --2- 172.21.15.32:0/1163791704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46e80cd910 0x7f46e8154210 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:06.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 >> 172.21.15.32:0/1163791704 conn(0x7f46e80d28b0 msgr2=0x7f46e80b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:06.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 shutdown_connections 2023-10-16T08:18:06.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:06.913+0000 7f46ee032640 1 -- 172.21.15.32:0/1163791704 wait complete. 2023-10-16T08:18:06.939 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1e4ad640 1 Processor -- start 2023-10-16T08:18:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1e4ad640 1 -- start start 2023-10-16T08:18:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1e4ad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b180d1490 0x7f0b1806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1e4ad640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180d3510 con 0x7f0b1815c980 2023-10-16T08:18:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1e4ad640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180d2f50 con 0x7f0b1815cd60 2023-10-16T08:18:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1e4ad640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b180cd2b0 con 0x7f0b180d1490 2023-10-16T08:18:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1ccaa640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b180d1490 0x7f0b1806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1ccaa640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b180d1490 0x7f0b1806af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40512/0 (socket says 172.21.15.32:40512) 2023-10-16T08:18:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/2002330242 learned_addr learned my addr 172.21.15.32:0/2002330242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/2002330242 >> v1:172.21.15.186:6789/0 conn(0x7f0b1815cd60 legacy=0x7f0b180cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/2002330242 >> v1:172.21.15.32:6789/0 conn(0x7f0b1815c980 legacy=0x7f0b180d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/2002330242 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180d3230 con 0x7f0b180d1490 2023-10-16T08:18:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.077+0000 7f0b0ffff640 1 -- 172.21.15.32:0/2002330242 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562638807 0 0) 0x7f0b14001190 con 0x7f0b1815c980 2023-10-16T08:18:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1ccaa640 1 --2- 172.21.15.32:0/2002330242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180d1490 0x7f0b1806af90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009870 tx=0x7f0b0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63694c439e794ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b0ffff640 1 -- 172.21.15.32:0/2002330242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080a6040 con 0x7f0b180d1490 2023-10-16T08:18:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b0ffff640 1 -- 172.21.15.32:0/2002330242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b080aa600 con 0x7f0b180d1490 2023-10-16T08:18:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b0ffff640 1 -- 172.21.15.32:0/2002330242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080aa990 con 0x7f0b180d1490 2023-10-16T08:18:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/2002330242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180d1490 msgr2=0x7f0b1806af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/2002330242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180d1490 0x7f0b1806af90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009870 tx=0x7f0b0801c480 comp rx=0 tx=0).stop 2023-10-16T08:18:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/2002330242 shutdown_connections 2023-10-16T08:18:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/2002330242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180d1490 0x7f0b1806af90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/2002330242 >> 172.21.15.32:0/2002330242 conn(0x7f0b1809ba80 msgr2=0x7f0b180d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/2002330242 shutdown_connections 2023-10-16T08:18:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.081+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/2002330242 wait complete. 2023-10-16T08:18:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 Processor -- start 2023-10-16T08:18:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 -- start start 2023-10-16T08:18:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b180d1490 0x7f0b18157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 0x7f0b180b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b1815cd60 0x7f0b180cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180cbf20 con 0x7f0b1815c980 2023-10-16T08:18:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180b28b0 con 0x7f0b180d1490 2023-10-16T08:18:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180b2b90 con 0x7f0b1815cd60 2023-10-16T08:18:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 0x7f0b180b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 0x7f0b180b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40516/0 (socket says 172.21.15.32:40516) 2023-10-16T08:18:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/817016355 learned_addr learned my addr 172.21.15.32:0/817016355 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1d4ab640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b180d1490 0x7f0b18157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1dcac640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b1815cd60 0x7f0b180cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b1815cd60 msgr2=0x7f0b180cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b1815cd60 0x7f0b180cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b180d1490 msgr2=0x7f0b18157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b180d1490 0x7f0b18157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 -- 172.21.15.32:0/817016355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080094d0 con 0x7f0b1815c980 2023-10-16T08:18:07.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1ccaa640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 0x7f0b180b21b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0b08019090 tx=0x7f0b0801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:07.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080aade0 con 0x7f0b1815c980 2023-10-16T08:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b080b7070 con 0x7f0b1815c980 2023-10-16T08:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080bd600 con 0x7f0b1815c980 2023-10-16T08:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b180b2e70 con 0x7f0b1815c980 2023-10-16T08:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.085+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b180b3150 con 0x7f0b1815c980 2023-10-16T08:18:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.089+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b18158f00 con 0x7f0b1815c980 2023-10-16T08:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.089+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0b080c6430 con 0x7f0b1815c980 2023-10-16T08:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.089+0000 7f0b0dffb640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0afc0664a0 0x7f0afc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.089+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f0b0811e550 con 0x7f0b1815c980 2023-10-16T08:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.089+0000 7f0b1d4ab640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0afc0664a0 0x7f0afc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.089+0000 7f0b1d4ab640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0afc0664a0 0x7f0afc068960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b140013f0 tx=0x7f0b14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.093+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0b18158f00 con 0x7f0b1815c980 2023-10-16T08:18:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b18158f00 con 0x7f0b1815c980 2023-10-16T08:18:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b0dffb640 1 -- 172.21.15.32:0/817016355 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0b18158f00 con 0x7f0b1815c980 2023-10-16T08:18:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:07.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0afc0664a0 msgr2=0x7f0afc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0afc0664a0 0x7f0afc068960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b140013f0 tx=0x7f0b14009040 comp rx=0 tx=0).stop 2023-10-16T08:18:07.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 msgr2=0x7f0b180b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 0x7f0b180b21b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0b08019090 tx=0x7f0b0801acc0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 shutdown_connections 2023-10-16T08:18:07.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0afc0664a0 0x7f0afc068960 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0b1815cd60 0x7f0b180cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b1815c980 0x7f0b180b21b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 --2- 172.21.15.32:0/817016355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b180d1490 0x7f0b18157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 >> 172.21.15.32:0/817016355 conn(0x7f0b1809ba80 msgr2=0x7f0b180c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 shutdown_connections 2023-10-16T08:18:07.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.265+0000 7f0b1e4ad640 1 -- 172.21.15.32:0/817016355 wait complete. 2023-10-16T08:18:07.291 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:18:07.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:07.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:07.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48255]: fuse finished with error 0 and tester_r 0 2023-10-16T08:18:07.313 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:07.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:18:07.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:07.405 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:18:07.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:18:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f713b53e640 1 Processor -- start 2023-10-16T08:18:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f713b53e640 1 -- start start 2023-10-16T08:18:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f713b53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 0x7f71340cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f713b53e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340ce770 con 0x7f71340d15d0 2023-10-16T08:18:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f713b53e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340d0600 con 0x7f71340d3010 2023-10-16T08:18:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f713b53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c6450 con 0x7f713415cb90 2023-10-16T08:18:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f7139adb640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71340d15d0 0x7f71340c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59184/0 (socket says 172.21.15.32:59184) 2023-10-16T08:18:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f7139adb640 1 -- 172.21.15.32:0/592861447 learned_addr learned my addr 172.21.15.32:0/592861447 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f71392da640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 0x7f71340cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997287742 0 0) 0x7f71340ce770 con 0x7f71340d15d0 2023-10-16T08:18:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7120003530 con 0x7f71340d15d0 2023-10-16T08:18:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 770962393 0 0) 0x7f7120003530 con 0x7f71340d15d0 2023-10-16T08:18:07.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 msgr2=0x7f71340cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 0x7f71340cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 >> v1:172.21.15.186:6789/0 conn(0x7f71340d3010 legacy=0x7f71340d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340d08a0 con 0x7f71340d15d0 2023-10-16T08:18:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f712c001a00 con 0x7f71340d15d0 2023-10-16T08:18:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 >> v1:172.21.15.32:6789/0 conn(0x7f71340d15d0 legacy=0x7f71340c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71200039b0 0x7f7120003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 0x7f71200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120004d10 0x7f7120005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c6450 con 0x7f7120004d10 2023-10-16T08:18:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712000bac0 con 0x7f71200039b0 2023-10-16T08:18:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.597+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712000bc80 con 0x7f713415cb90 2023-10-16T08:18:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120004d10 0x7f7120005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f71392da640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71200039b0 0x7f7120003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7138ad9640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 0x7f71200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 -- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 msgr2=0x7f71200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:07.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f713415cb90 0x7f71200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 -- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71200039b0 msgr2=0x7f7120003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71200039b0 0x7f7120003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 -- 172.21.15.32:0/592861447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120003530 con 0x7f7120004d10 2023-10-16T08:18:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f7139adb640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120004d10 0x7f7120005190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f712c00a800 tx=0x7f712c01fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f1e7a0e1771f5c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0aa690 con 0x7f7120004d10 2023-10-16T08:18:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c0aa7f0 con 0x7f7120004d10 2023-10-16T08:18:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.601+0000 7f712bfff640 1 -- 172.21.15.32:0/592861447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0aabe0 con 0x7f7120004d10 2023-10-16T08:18:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- 172.21.15.32:0/592861447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120004d10 msgr2=0x7f7120005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 --2- 172.21.15.32:0/592861447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7120004d10 0x7f7120005190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f712c00a800 tx=0x7f712c01fa60 comp rx=0 tx=0).stop 2023-10-16T08:18:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7139adb640 1 -- 172.21.15.32:0/592861447 reap_dead start 2023-10-16T08:18:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- 172.21.15.32:0/592861447 shutdown_connections 2023-10-16T08:18:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- 172.21.15.32:0/592861447 >> 172.21.15.32:0/592861447 conn(0x7f713409bc50 msgr2=0x7f71340d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- 172.21.15.32:0/592861447 shutdown_connections 2023-10-16T08:18:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- 172.21.15.32:0/592861447 wait complete. 2023-10-16T08:18:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 Processor -- start 2023-10-16T08:18:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- start start 2023-10-16T08:18:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7120004d10 0x7f713415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 0x7f713406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713406c880 0x7f71341544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713406cd00 con 0x7f71341f63a0 2023-10-16T08:18:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b6620 con 0x7f713406c880 2023-10-16T08:18:07.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f713b53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b5a10 con 0x7f7120004d10 2023-10-16T08:18:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 0x7f713406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 0x7f713406c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40526/0 (socket says 172.21.15.32:40526) 2023-10-16T08:18:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 -- 172.21.15.32:0/2408178276 learned_addr learned my addr 172.21.15.32:0/2408178276 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f71392da640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7120004d10 0x7f713415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7139adb640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713406c880 0x7f71341544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 -- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7120004d10 msgr2=0x7f713415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7120004d10 0x7f713415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 -- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713406c880 msgr2=0x7f71341544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713406c880 0x7f71341544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.605+0000 7f7138ad9640 1 -- 172.21.15.32:0/2408178276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340b5cf0 con 0x7f71341f63a0 2023-10-16T08:18:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7138ad9640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 0x7f713406c340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f712400f5e0 tx=0x7f712400f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240052e0 con 0x7f71341f63a0 2023-10-16T08:18:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124005440 con 0x7f71341f63a0 2023-10-16T08:18:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340b5fd0 con 0x7f71341f63a0 2023-10-16T08:18:07.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124003820 con 0x7f71341f63a0 2023-10-16T08:18:07.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71340caa50 con 0x7f71341f63a0 2023-10-16T08:18:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7134074010 con 0x7f71341f63a0 2023-10-16T08:18:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7124003980 con 0x7f71341f63a0 2023-10-16T08:18:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7129ffb640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71180664a0 0x7f7118068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.609+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f7124083ea0 con 0x7f71341f63a0 2023-10-16T08:18:07.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.613+0000 7f71392da640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71180664a0 0x7f7118068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.613+0000 7f71392da640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71180664a0 0x7f7118068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f71340c7930 tx=0x7f7130006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:07.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.613+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7124051020 con 0x7f71341f63a0 2023-10-16T08:18:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7134155350 con 0x7f71341f63a0 2023-10-16T08:18:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f7129ffb640 1 -- 172.21.15.32:0/2408178276 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+97 (secure 0 0 0) 0x7f7124054640 con 0x7f71341f63a0 2023-10-16T08:18:07.815 INFO:teuthology.orchestra.run.smithi032.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data data_new ] 2023-10-16T08:18:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71180664a0 msgr2=0x7f7118068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71180664a0 0x7f7118068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f71340c7930 tx=0x7f7130006f90 comp rx=0 tx=0).stop 2023-10-16T08:18:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 msgr2=0x7f713406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 0x7f713406c340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f712400f5e0 tx=0x7f712400f9e0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 shutdown_connections 2023-10-16T08:18:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f713406c880 0x7f71341544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71341f63a0 0x7f713406c340 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71180664a0 0x7f7118068960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 --2- 172.21.15.32:0/2408178276 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7120004d10 0x7f713415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:07.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 >> 172.21.15.32:0/2408178276 conn(0x7f713409bc50 msgr2=0x7f713406ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:07.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 shutdown_connections 2023-10-16T08:18:07.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:07.809+0000 7f713b53e640 1 -- 172.21.15.32:0/2408178276 wait complete. 2023-10-16T08:18:07.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:18:07.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:18:07.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:18:07.837 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:18:07.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:07.837 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:18:07.838 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:18:07.838 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:18:07.838 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:07.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:07.839 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:18:07.849 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:18:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:18:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:18:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:18:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.850 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:18:07.851 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:18:07.851 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:18:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:18:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:18:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42317sec preferred_lft 42317sec 2023-10-16T08:18:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:18:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.853 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:18:07.853 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:18:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:18:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.854 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:18:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:18:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:18:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.855 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:18:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:18:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.856 INFO:teuthology.orchestra.run.smithi032.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:18:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T08:18:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:07.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:07.858 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:18:07.912 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:18:07.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:07.915 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:18:07.967 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:18:07.967 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:18:07.967 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:18:07.967 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:18:07.968 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:18:07.968 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:18:07.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:07.969 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:18:07.969 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:18:07.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:18:07.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:18:07.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:18:07.971 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:18:08.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.062 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:18:08.072 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:18:08.072 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:18:08.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:08.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:18:08.138 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:18:08.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:18:08.138 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:08.194 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:08.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:18:08.195 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:08.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:18:08.310 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:18:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:18:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:18:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:18:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:18:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:18:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:18:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:18:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:18:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:18:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:18:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:18:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:18:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:18:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:18:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:18:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:18:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:18:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:18:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:18:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:18:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:18:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:18:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:18:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:18:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:18:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:18:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:18:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:18:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:18:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:08.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.371 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:18:08.421 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:18:08.421 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:18:08.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:18:08.490 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:18:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:18:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:18:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:18:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:18:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:18:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:18:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:18:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:18:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:18:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:18:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:18:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:18:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:18:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:18:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:18:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:18:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:18:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:18:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:18:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:18:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:18:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:18:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:18:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:18:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:18:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:18:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:18:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:18:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:18:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:08.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.550 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:18:08.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:18:08.561+0000 7fde51b93580 -1 init, newargv = 0x5650d462ebe0 newargc=15 2023-10-16T08:18:08.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48813]: starting ceph client 2023-10-16T08:18:08.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48813]: starting fuse 2023-10-16T08:18:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:18:08.576 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:18:08.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:18:08.577 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:18:08.577 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:18:08.578 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:18:08.579 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:18:08.579 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:08.580 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:18:08.580 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:18:08.580 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:18:08.581 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:18:08.581 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:08.581 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:18:08.582 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:18:08.582 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:18:08.582 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:18:08.583 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:08.583 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:18:08.584 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:18:08.584 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:18:08.584 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:18:08.585 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:18:08.585 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:18:08.586 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:18:08.586 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:18:08.586 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:08.587 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:18:08.587 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:18:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.48813.asok 2023-10-16T08:18:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:18:08.711 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48813.asok 2023-10-16T08:18:08.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48813.asok status 2023-10-16T08:18:08.891 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:18:08.891 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:18:08.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:18:08.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:18:08.892 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:18:08.892 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:18:08.892 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:18:08.892 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "48813", 2023-10-16T08:18:08.892 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:18:08.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:08.893 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:18:08.893 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:18:08.893 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5829, 2023-10-16T08:18:08.893 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:18:08.894 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:18:08.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:18:08.894 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 5829 2023-10-16T08:18:08.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:08.894 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:18:08.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:18:08.895 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:18:08.895 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1557648857 2023-10-16T08:18:08.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:08.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:08.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:18:08.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:18:08.896 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:18:08.896 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1557648857 2023-10-16T08:18:08.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:08.897 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.5829 192.168.0.1:0/1557648857", 2023-10-16T08:18:08.897 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1557648857", 2023-10-16T08:18:08.897 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:18:08.897 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 19, 2023-10-16T08:18:08.898 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 42, 2023-10-16T08:18:08.898 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:18:08.898 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:18:08.898 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1" 2023-10-16T08:18:08.898 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:18:08.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.906 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:18:08.907 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:08.916 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:18:08.917 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:08.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:08.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:08.978 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-10-16T08:18:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:setfattr: layoutfile: Permission denied 2023-10-16T08:18:09.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:18:09.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:09.033 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-10-16T08:18:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:setfattr: layoutdir: Permission denied 2023-10-16T08:18:09.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:18:09.089 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:09.089 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:09.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.325+0000 7f421f49e640 1 Processor -- start 2023-10-16T08:18:09.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.325+0000 7f421f49e640 1 -- start start 2023-10-16T08:18:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.325+0000 7f421f49e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4218070c40 0x7f421815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.325+0000 7f421f49e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f42180b07f0 0x7f4218164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.325+0000 7f421f49e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421815be60 con 0x7f42180b4020 2023-10-16T08:18:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421f49e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421815b3b0 con 0x7f4218070c40 2023-10-16T08:18:09.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421f49e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421815c1f0 con 0x7f42180b07f0 2023-10-16T08:18:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4218070c40 0x7f421815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4218070c40 0x7f421815ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40536/0 (socket says 172.21.15.32:40536) 2023-10-16T08:18:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 -- 172.21.15.32:0/1220098836 learned_addr learned my addr 172.21.15.32:0/1220098836 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:09.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 -- 172.21.15.32:0/1220098836 >> v2:172.21.15.186:3300/0 conn(0x7f42180b07f0 msgr2=0x7f4218164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 --2- 172.21.15.32:0/1220098836 >> v2:172.21.15.186:3300/0 conn(0x7f42180b07f0 0x7f4218164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 -- 172.21.15.32:0/1220098836 >> v1:172.21.15.186:6789/0 conn(0x7f42180b4020 legacy=0x7f42180b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:09.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 -- 172.21.15.32:0/1220098836 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4218071020 con 0x7f4218070c40 2023-10-16T08:18:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f421d23a640 1 --2- 172.21.15.32:0/1220098836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4218070c40 0x7f421815ae70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f421806e590 tx=0x7f421401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85284eac1e21b50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f420ffff640 1 -- 172.21.15.32:0/1220098836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140a7070 con 0x7f4218070c40 2023-10-16T08:18:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f420ffff640 1 -- 172.21.15.32:0/1220098836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42140ab430 con 0x7f4218070c40 2023-10-16T08:18:09.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.329+0000 7f420ffff640 1 -- 172.21.15.32:0/1220098836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140ab7c0 con 0x7f4218070c40 2023-10-16T08:18:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- 172.21.15.32:0/1220098836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4218070c40 msgr2=0x7f421815ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 --2- 172.21.15.32:0/1220098836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4218070c40 0x7f421815ae70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f421806e590 tx=0x7f421401c430 comp rx=0 tx=0).stop 2023-10-16T08:18:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- 172.21.15.32:0/1220098836 shutdown_connections 2023-10-16T08:18:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 --2- 172.21.15.32:0/1220098836 >> v2:172.21.15.186:3300/0 conn(0x7f42180b07f0 0x7f4218164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 --2- 172.21.15.32:0/1220098836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4218070c40 0x7f421815ae70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- 172.21.15.32:0/1220098836 >> 172.21.15.32:0/1220098836 conn(0x7f421806c190 msgr2=0x7f421806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- 172.21.15.32:0/1220098836 shutdown_connections 2023-10-16T08:18:09.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- 172.21.15.32:0/1220098836 wait complete. 2023-10-16T08:18:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 Processor -- start 2023-10-16T08:18:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- start start 2023-10-16T08:18:09.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4218070c40 0x7f42180cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42180b07f0 0x7f42181449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 0x7f42180c2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218159940 con 0x7f42180b4020 2023-10-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218154d40 con 0x7f4218070c40 2023-10-16T08:18:09.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421f49e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180afe30 con 0x7f42180b07f0 2023-10-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421da3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 0x7f42180c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421da3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 0x7f42180c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40546/0 (socket says 172.21.15.32:40546) 2023-10-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421da3b640 1 -- 172.21.15.32:0/1206213942 learned_addr learned my addr 172.21.15.32:0/1206213942 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421d23a640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4218070c40 0x7f42180cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421ca39640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42180b07f0 0x7f42181449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.333+0000 7f421da3b640 1 -- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42180b07f0 msgr2=0x7f42181449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421da3b640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42180b07f0 0x7f42181449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421da3b640 1 -- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4218070c40 msgr2=0x7f42180cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421da3b640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4218070c40 0x7f42180cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421da3b640 1 -- 172.21.15.32:0/1206213942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140094d0 con 0x7f42180b4020 2023-10-16T08:18:09.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421da3b640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 0x7f42180c2190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f421000b9e0 tx=0x7f421000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:09.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000caa0 con 0x7f42180b4020 2023-10-16T08:18:09.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421000cc20 con 0x7f42180b4020 2023-10-16T08:18:09.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42181532d0 con 0x7f42180b4020 2023-10-16T08:18:09.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210014440 con 0x7f42180b4020 2023-10-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42180b03b0 con 0x7f42180b4020 2023-10-16T08:18:09.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4218159af0 con 0x7f42180b4020 2023-10-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4210007370 con 0x7f42180b4020 2023-10-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f420dffb640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41fc0664a0 0x7f41fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f42100840a0 con 0x7f42180b4020 2023-10-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.337+0000 7f421d23a640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41fc0664a0 0x7f41fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.341+0000 7f421d23a640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41fc0664a0 0x7f41fc068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f42140af000 tx=0x7f4214008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:09.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.341+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4218159af0 con 0x7f42180b4020 2023-10-16T08:18:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.501+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4218159af0 con 0x7f42180b4020 2023-10-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.501+0000 7f420dffb640 1 -- 172.21.15.32:0/1206213942 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7f4218159af0 con 0x7f42180b4020 2023-10-16T08:18:09.508 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:09.509 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41fc0664a0 msgr2=0x7f41fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41fc0664a0 0x7f41fc068960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f42140af000 tx=0x7f4214008040 comp rx=0 tx=0).stop 2023-10-16T08:18:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 msgr2=0x7f42180c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 0x7f42180c2190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f421000b9e0 tx=0x7f421000bea0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 shutdown_connections 2023-10-16T08:18:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42180b4020 0x7f42180c2190 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f41fc0664a0 0x7f41fc068960 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f42180b07f0 0x7f42181449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 --2- 172.21.15.32:0/1206213942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4218070c40 0x7f42180cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 >> 172.21.15.32:0/1206213942 conn(0x7f421806c190 msgr2=0x7f42180ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 shutdown_connections 2023-10-16T08:18:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.505+0000 7f421f49e640 1 -- 172.21.15.32:0/1206213942 wait complete. 2023-10-16T08:18:09.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:09.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.669+0000 7f87938d6640 1 Processor -- start 2023-10-16T08:18:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.669+0000 7f87938d6640 1 -- start start 2023-10-16T08:18:09.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f87938d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c159900 0x7f878c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f87938d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f878c06a5a0 0x7f878c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f87938d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c0b4360 con 0x7f878c0b3f00 2023-10-16T08:18:09.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f87938d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c0cd740 con 0x7f878c06a5a0 2023-10-16T08:18:09.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f87938d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b4600 con 0x7f878c159900 2023-10-16T08:18:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f878c06a5a0 0x7f878c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f878c06a5a0 0x7f878c0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40550/0 (socket says 172.21.15.32:40550) 2023-10-16T08:18:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 -- 172.21.15.32:0/3896642940 learned_addr learned my addr 172.21.15.32:0/3896642940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791672640 1 --2- 172.21.15.32:0/3896642940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c159900 0x7f878c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 -- 172.21.15.32:0/3896642940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c159900 msgr2=0x7f878c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 --2- 172.21.15.32:0/3896642940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c159900 0x7f878c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 -- 172.21.15.32:0/3896642940 >> v1:172.21.15.32:6789/0 conn(0x7f878c0b3f00 legacy=0x7f878c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8791e73640 1 -- 172.21.15.32:0/3896642940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0c8dd0 con 0x7f878c06a5a0 2023-10-16T08:18:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.673+0000 7f8783fff640 1 -- 172.21.15.32:0/3896642940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215954641 0 0) 0x7f878c0b4360 con 0x7f878c0b3f00 2023-10-16T08:18:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f8791e73640 1 --2- 172.21.15.32:0/3896642940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c9b90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8784009e50 tx=0x7f878401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9782b63daf42677 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f8783fff640 1 -- 172.21.15.32:0/3896642940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840a7ac0 con 0x7f878c06a5a0 2023-10-16T08:18:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f8783fff640 1 -- 172.21.15.32:0/3896642940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87840a7c20 con 0x7f878c06a5a0 2023-10-16T08:18:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f8783fff640 1 -- 172.21.15.32:0/3896642940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840af3f0 con 0x7f878c06a5a0 2023-10-16T08:18:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 -- 172.21.15.32:0/3896642940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 msgr2=0x7f878c0c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 --2- 172.21.15.32:0/3896642940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c9b90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8784009e50 tx=0x7f878401ebf0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 -- 172.21.15.32:0/3896642940 shutdown_connections 2023-10-16T08:18:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 --2- 172.21.15.32:0/3896642940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c9b90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 --2- 172.21.15.32:0/3896642940 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c159900 0x7f878c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 -- 172.21.15.32:0/3896642940 >> 172.21.15.32:0/3896642940 conn(0x7f878c0b0da0 msgr2=0x7f878c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 -- 172.21.15.32:0/3896642940 shutdown_connections 2023-10-16T08:18:09.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 -- 172.21.15.32:0/3896642940 wait complete. 2023-10-16T08:18:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 Processor -- start 2023-10-16T08:18:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.677+0000 7f87938d6640 1 -- start start 2023-10-16T08:18:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c0b3f00 0x7f878c0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c159900 0x7f878c070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0c1ef0 con 0x7f878c06a5a0 2023-10-16T08:18:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c152870 con 0x7f878c159900 2023-10-16T08:18:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0c67e0 con 0x7f878c0b3f00 2023-10-16T08:18:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40556/0 (socket says 172.21.15.32:40556) 2023-10-16T08:18:09.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 -- 172.21.15.32:0/1816863486 learned_addr learned my addr 172.21.15.32:0/1816863486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791e73640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c159900 0x7f878c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8790e71640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c0b3f00 0x7f878c0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 -- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c0b3f00 msgr2=0x7f878c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c0b3f00 0x7f878c0701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 -- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c159900 msgr2=0x7f878c070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c159900 0x7f878c070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 -- 172.21.15.32:0/1816863486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784009b30 con 0x7f878c06a5a0 2023-10-16T08:18:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791e73640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c159900 0x7f878c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:09.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8790e71640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c0b3f00 0x7f878c0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:09.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8791672640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c8300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f878800d5f0 tx=0x7f878800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:09.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880125f0 con 0x7f878c06a5a0 2023-10-16T08:18:09.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788012750 con 0x7f878c06a5a0 2023-10-16T08:18:09.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788012b00 con 0x7f878c06a5a0 2023-10-16T08:18:09.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c0c6ac0 con 0x7f878c06a5a0 2023-10-16T08:18:09.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c0c6470 con 0x7f878c06a5a0 2023-10-16T08:18:09.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c04e170 con 0x7f878c06a5a0 2023-10-16T08:18:09.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.681+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8788012c60 con 0x7f878c06a5a0 2023-10-16T08:18:09.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.685+0000 7f8781ffb640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87700664d0 0x7f8770068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:09.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.685+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f8788084640 con 0x7f878c06a5a0 2023-10-16T08:18:09.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.685+0000 7f8790e71640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87700664d0 0x7f8770068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:09.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.685+0000 7f8790e71640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87700664d0 0x7f8770068990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f878c1505c0 tx=0x7f877c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:09.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.689+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f87880517d0 con 0x7f878c06a5a0 2023-10-16T08:18:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.845+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f878c154bf0 con 0x7f878c06a5a0 2023-10-16T08:18:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.845+0000 7f8781ffb640 1 -- 172.21.15.32:0/1816863486 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8788054df0 con 0x7f878c06a5a0 2023-10-16T08:18:09.853 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87700664d0 msgr2=0x7f8770068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87700664d0 0x7f8770068990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f878c1505c0 tx=0x7f877c009040 comp rx=0 tx=0).stop 2023-10-16T08:18:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 msgr2=0x7f878c0c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c8300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f878800d5f0 tx=0x7f878800dab0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 shutdown_connections 2023-10-16T08:18:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f878c159900 0x7f878c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f87700664d0 0x7f8770068990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f878c0b3f00 0x7f878c0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 --2- 172.21.15.32:0/1816863486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878c06a5a0 0x7f878c0c8300 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 >> 172.21.15.32:0/1816863486 conn(0x7f878c0b0da0 msgr2=0x7f878c0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 shutdown_connections 2023-10-16T08:18:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:09.849+0000 7f87938d6640 1 -- 172.21.15.32:0/1816863486 wait complete. 2023-10-16T08:18:09.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.013+0000 7f183c805640 1 Processor -- start 2023-10-16T08:18:10.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.013+0000 7f183c805640 1 -- start start 2023-10-16T08:18:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.013+0000 7f183c805640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1838159900 0x7f183806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.013+0000 7f183c805640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380b4390 con 0x7f18380b3f00 2023-10-16T08:18:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.013+0000 7f183c805640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380b4600 con 0x7f183806a5a0 2023-10-16T08:18:10.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.013+0000 7f183c805640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380cd740 con 0x7f1838159900 2023-10-16T08:18:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1838159900 0x7f183806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1838159900 0x7f183806e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40570/0 (socket says 172.21.15.32:40570) 2023-10-16T08:18:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f182ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18380b3f00 0x7f1838069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59208/0 (socket says 172.21.15.32:59208) 2023-10-16T08:18:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 -- 172.21.15.32:0/4194248765 learned_addr learned my addr 172.21.15.32:0/4194248765 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 -- 172.21.15.32:0/4194248765 >> v1:172.21.15.186:6789/0 conn(0x7f183806a5a0 legacy=0x7f18380c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 -- 172.21.15.32:0/4194248765 >> v1:172.21.15.32:6789/0 conn(0x7f18380b3f00 legacy=0x7f1838069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 -- 172.21.15.32:0/4194248765 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380c8dd0 con 0x7f1838159900 2023-10-16T08:18:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f183659c640 1 --2- 172.21.15.32:0/4194248765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1838159900 0x7f183806e260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1828009870 tx=0x7f182801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eebf0ae0c63568 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f1835d9b640 1 -- 172.21.15.32:0/4194248765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280aa430 con 0x7f1838159900 2023-10-16T08:18:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f1835d9b640 1 -- 172.21.15.32:0/4194248765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18280aa590 con 0x7f1838159900 2023-10-16T08:18:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.017+0000 7f1835d9b640 1 -- 172.21.15.32:0/4194248765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280aa920 con 0x7f1838159900 2023-10-16T08:18:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- 172.21.15.32:0/4194248765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1838159900 msgr2=0x7f183806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 --2- 172.21.15.32:0/4194248765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1838159900 0x7f183806e260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1828009870 tx=0x7f182801c4d0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- 172.21.15.32:0/4194248765 shutdown_connections 2023-10-16T08:18:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 --2- 172.21.15.32:0/4194248765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1838159900 0x7f183806e260 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- 172.21.15.32:0/4194248765 >> 172.21.15.32:0/4194248765 conn(0x7f18380b0da0 msgr2=0x7f18380ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- 172.21.15.32:0/4194248765 shutdown_connections 2023-10-16T08:18:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- 172.21.15.32:0/4194248765 wait complete. 2023-10-16T08:18:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 Processor -- start 2023-10-16T08:18:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- start start 2023-10-16T08:18:10.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806a5a0 0x7f1838168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 0x7f18380aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1838159900 0x7f18380af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381f8460 con 0x7f18380b3f00 2023-10-16T08:18:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381f8720 con 0x7f1838159900 2023-10-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183c805640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18381f8a00 con 0x7f183806a5a0 2023-10-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f182ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 0x7f18380aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f182ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 0x7f18380aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40586/0 (socket says 172.21.15.32:40586) 2023-10-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f182ffff640 1 -- 172.21.15.32:0/3906448617 learned_addr learned my addr 172.21.15.32:0/3906448617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f183659c640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806a5a0 0x7f1838168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.021+0000 7f1836d9d640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1838159900 0x7f18380af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182ffff640 1 -- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806a5a0 msgr2=0x7f1838168c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182ffff640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806a5a0 0x7f1838168c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182ffff640 1 -- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1838159900 msgr2=0x7f18380af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182ffff640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1838159900 0x7f18380af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182ffff640 1 -- 172.21.15.32:0/3906448617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280094d0 con 0x7f18380b3f00 2023-10-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182ffff640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 0x7f18380aedf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f182000c490 tx=0x7f182000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820007480 con 0x7f18380b3f00 2023-10-16T08:18:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18200075e0 con 0x7f18380b3f00 2023-10-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820004a10 con 0x7f18380b3f00 2023-10-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18381f8ce0 con 0x7f18380b3f00 2023-10-16T08:18:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1838169140 con 0x7f18380b3f00 2023-10-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183804e170 con 0x7f18380b3f00 2023-10-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.025+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1820004b70 con 0x7f18380b3f00 2023-10-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.029+0000 7f182effd640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f18180664a0 0x7f1818068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.029+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f1820083310 con 0x7f18380b3f00 2023-10-16T08:18:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.029+0000 7f183659c640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f18180664a0 0x7f1818068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.029+0000 7f183659c640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f18180664a0 0x7f1818068960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f18280ae000 tx=0x7f18280b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.033+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f18200504b0 con 0x7f18380b3f00 2023-10-16T08:18:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.185+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1838154460 con 0x7f18380b3f00 2023-10-16T08:18:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.185+0000 7f182effd640 1 -- 172.21.15.32:0/3906448617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7f1820053ad0 con 0x7f18380b3f00 2023-10-16T08:18:10.193 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f18180664a0 msgr2=0x7f1818068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f18180664a0 0x7f1818068960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f18280ae000 tx=0x7f18280b5040 comp rx=0 tx=0).stop 2023-10-16T08:18:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 msgr2=0x7f18380aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 0x7f18380aedf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f182000c490 tx=0x7f182000c950 comp rx=0 tx=0).stop 2023-10-16T08:18:10.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 shutdown_connections 2023-10-16T08:18:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1838159900 0x7f18380af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18380b3f00 0x7f18380aedf0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f18180664a0 0x7f1818068960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 --2- 172.21.15.32:0/3906448617 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f183806a5a0 0x7f1838168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 >> 172.21.15.32:0/3906448617 conn(0x7f18380b0da0 msgr2=0x7f18380ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 shutdown_connections 2023-10-16T08:18:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.189+0000 7f183c805640 1 -- 172.21.15.32:0/3906448617 wait complete. 2023-10-16T08:18:10.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.353+0000 7f806aa68640 1 Processor -- start 2023-10-16T08:18:10.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.353+0000 7f806aa68640 1 -- start start 2023-10-16T08:18:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f806aa68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f806406df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f806aa68640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80640c8dd0 0x7f80640702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f806aa68640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8064070830 0x7f806415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f806aa68640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8064070cc0 con 0x7f80640c8dd0 2023-10-16T08:18:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f806aa68640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80640c9600 con 0x7f8064070830 2023-10-16T08:18:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f806aa68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80640c98e0 con 0x7f80641590d0 2023-10-16T08:18:10.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80640c8dd0 0x7f80640702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80640c8dd0 0x7f80640702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40596/0 (socket says 172.21.15.32:40596) 2023-10-16T08:18:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 -- 172.21.15.32:0/1610577645 learned_addr learned my addr 172.21.15.32:0/1610577645 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8068804640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f806406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8069005640 1 --2- 172.21.15.32:0/1610577645 >> v2:172.21.15.186:3300/0 conn(0x7f8064070830 0x7f806415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 -- 172.21.15.32:0/1610577645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 msgr2=0x7f806406df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f806406df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 -- 172.21.15.32:0/1610577645 >> v2:172.21.15.186:3300/0 conn(0x7f8064070830 msgr2=0x7f806415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 --2- 172.21.15.32:0/1610577645 >> v2:172.21.15.186:3300/0 conn(0x7f8064070830 0x7f806415cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8063fff640 1 -- 172.21.15.32:0/1610577645 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80640c9bc0 con 0x7f80640c8dd0 2023-10-16T08:18:10.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8068804640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f806406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.357+0000 7f8069005640 1 --2- 172.21.15.32:0/1610577645 >> v2:172.21.15.186:3300/0 conn(0x7f8064070830 0x7f806415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f8063fff640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f80640702f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8064071750 tx=0x7f805001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fdae6a6d98639c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f80637fe640 1 -- 172.21.15.32:0/1610577645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500ab610 con 0x7f80640c8dd0 2023-10-16T08:18:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f80637fe640 1 -- 172.21.15.32:0/1610577645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80500ab770 con 0x7f80640c8dd0 2023-10-16T08:18:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f80637fe640 1 -- 172.21.15.32:0/1610577645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500abb00 con 0x7f80640c8dd0 2023-10-16T08:18:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 -- 172.21.15.32:0/1610577645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 msgr2=0x7f80640702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f80640702f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8064071750 tx=0x7f805001c4d0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 -- 172.21.15.32:0/1610577645 shutdown_connections 2023-10-16T08:18:10.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 --2- 172.21.15.32:0/1610577645 >> v2:172.21.15.186:3300/0 conn(0x7f8064070830 0x7f806415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f80640702f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 --2- 172.21.15.32:0/1610577645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f806406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 -- 172.21.15.32:0/1610577645 >> 172.21.15.32:0/1610577645 conn(0x7f80640b0da0 msgr2=0x7f80640ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:10.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 -- 172.21.15.32:0/1610577645 shutdown_connections 2023-10-16T08:18:10.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 -- 172.21.15.32:0/1610577645 wait complete. 2023-10-16T08:18:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 Processor -- start 2023-10-16T08:18:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.361+0000 7f806aa68640 1 -- start start 2023-10-16T08:18:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8064070830 0x7f80640c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f8064145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f80640c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80641464e0 con 0x7f80640c8dd0 2023-10-16T08:18:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640c8620 con 0x7f8064070830 2023-10-16T08:18:10.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80640c5d90 con 0x7f80641590d0 2023-10-16T08:18:10.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f8064145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f8064145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40598/0 (socket says 172.21.15.32:40598) 2023-10-16T08:18:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 -- 172.21.15.32:0/2273507231 learned_addr learned my addr 172.21.15.32:0/2273507231 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8069005640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f80640c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8068804640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8064070830 0x7f80640c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 -- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 msgr2=0x7f80640c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f80640c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 -- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8064070830 msgr2=0x7f80640c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8064070830 0x7f80640c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 -- 172.21.15.32:0/2273507231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80500094d0 con 0x7f80640c8dd0 2023-10-16T08:18:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f8063fff640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f8064145fa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f805001ab80 tx=0x7f805001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500b4800 con 0x7f80640c8dd0 2023-10-16T08:18:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80500b4960 con 0x7f80640c8dd0 2023-10-16T08:18:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80640ca820 con 0x7f80640c8dd0 2023-10-16T08:18:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500b4cf0 con 0x7f80640c8dd0 2023-10-16T08:18:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80640c7e30 con 0x7f80640c8dd0 2023-10-16T08:18:10.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806404e170 con 0x7f80640c8dd0 2023-10-16T08:18:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.365+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f80500b3380 con 0x7f80640c8dd0 2023-10-16T08:18:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.369+0000 7f80617fa640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80480664a0 0x7f8048068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.369+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f805011e880 con 0x7f80640c8dd0 2023-10-16T08:18:10.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.369+0000 7f8068804640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80480664a0 0x7f8048068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.369+0000 7f8068804640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80480664a0 0x7f8048068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f805800ab60 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.373+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f80500ebaa0 con 0x7f80640c8dd0 2023-10-16T08:18:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.525+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f80640c8400 con 0x7f80640c8dd0 2023-10-16T08:18:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.525+0000 7f80617fa640 1 -- 172.21.15.32:0/2273507231 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7f80500c40b0 con 0x7f80640c8dd0 2023-10-16T08:18:10.534 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80480664a0 msgr2=0x7f8048068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80480664a0 0x7f8048068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f805800ab60 tx=0x7f8058008040 comp rx=0 tx=0).stop 2023-10-16T08:18:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 msgr2=0x7f8064145fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f8064145fa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f805001ab80 tx=0x7f805001ad10 comp rx=0 tx=0).stop 2023-10-16T08:18:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 shutdown_connections 2023-10-16T08:18:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f80480664a0 0x7f8048068960 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f80641590d0 0x7f80640c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80640c8dd0 0x7f8064145fa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 --2- 172.21.15.32:0/2273507231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8064070830 0x7f80640c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 >> 172.21.15.32:0/2273507231 conn(0x7f80640b0da0 msgr2=0x7f806406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 shutdown_connections 2023-10-16T08:18:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.529+0000 7f806aa68640 1 -- 172.21.15.32:0/2273507231 wait complete. 2023-10-16T08:18:10.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:10.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.693+0000 7fef3be46640 1 Processor -- start 2023-10-16T08:18:10.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.693+0000 7fef3be46640 1 -- start start 2023-10-16T08:18:10.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.693+0000 7fef3be46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef34159900 0x7fef3406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef3be46640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fef3406a5a0 0x7fef340c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef3be46640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef340b4360 con 0x7fef340b3f00 2023-10-16T08:18:10.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef3be46640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef340cd740 con 0x7fef3406a5a0 2023-10-16T08:18:10.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef3be46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340b4600 con 0x7fef34159900 2023-10-16T08:18:10.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef393e1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef340b3f00 0x7fef34069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59212/0 (socket says 172.21.15.32:59212) 2023-10-16T08:18:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef393e1640 1 -- 172.21.15.32:0/307128158 learned_addr learned my addr 172.21.15.32:0/307128158 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef39be2640 1 --2- 172.21.15.32:0/307128158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef34159900 0x7fef3406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef3a3e3640 1 --2- 172.21.15.32:0/307128158 >> v2:172.21.15.186:3300/0 conn(0x7fef3406a5a0 0x7fef340c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef38be0640 1 -- 172.21.15.32:0/307128158 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141320841 0 0) 0x7fef340b4360 con 0x7fef340b3f00 2023-10-16T08:18:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef38be0640 1 -- 172.21.15.32:0/307128158 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef20003430 con 0x7fef340b3f00 2023-10-16T08:18:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef39be2640 1 -- 172.21.15.32:0/307128158 >> v2:172.21.15.186:3300/0 conn(0x7fef3406a5a0 msgr2=0x7fef340c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef39be2640 1 --2- 172.21.15.32:0/307128158 >> v2:172.21.15.186:3300/0 conn(0x7fef3406a5a0 0x7fef340c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef39be2640 1 -- 172.21.15.32:0/307128158 >> v1:172.21.15.32:6789/0 conn(0x7fef340b3f00 legacy=0x7fef34069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef38be0640 1 -- 172.21.15.32:0/307128158 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215477122 0 0) 0x7fef20003430 con 0x7fef340b3f00 2023-10-16T08:18:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef39be2640 1 -- 172.21.15.32:0/307128158 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef340c8dd0 con 0x7fef34159900 2023-10-16T08:18:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.697+0000 7fef39be2640 1 --2- 172.21.15.32:0/307128158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef34159900 0x7fef3406e260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef300097c0 tx=0x7fef3001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc0c8f716933aca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef38be0640 1 -- 172.21.15.32:0/307128158 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24001680 con 0x7fef34159900 2023-10-16T08:18:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef38be0640 1 -- 172.21.15.32:0/307128158 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef300a7050 con 0x7fef34159900 2023-10-16T08:18:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef38be0640 1 -- 172.21.15.32:0/307128158 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300ab620 con 0x7fef34159900 2023-10-16T08:18:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- 172.21.15.32:0/307128158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef34159900 msgr2=0x7fef3406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 --2- 172.21.15.32:0/307128158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef34159900 0x7fef3406e260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef300097c0 tx=0x7fef3001c370 comp rx=0 tx=0).stop 2023-10-16T08:18:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- 172.21.15.32:0/307128158 shutdown_connections 2023-10-16T08:18:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 --2- 172.21.15.32:0/307128158 >> v2:172.21.15.186:3300/0 conn(0x7fef3406a5a0 0x7fef340c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 --2- 172.21.15.32:0/307128158 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef34159900 0x7fef3406e260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- 172.21.15.32:0/307128158 >> 172.21.15.32:0/307128158 conn(0x7fef340b0da0 msgr2=0x7fef340ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- 172.21.15.32:0/307128158 shutdown_connections 2023-10-16T08:18:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- 172.21.15.32:0/307128158 wait complete. 2023-10-16T08:18:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 Processor -- start 2023-10-16T08:18:10.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- start start 2023-10-16T08:18:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 0x7fef3414ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef340b3f00 0x7fef341548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef34159900 0x7fef34154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34158f00 con 0x7fef3406a5a0 2023-10-16T08:18:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c3e00 con 0x7fef34159900 2023-10-16T08:18:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.701+0000 7fef3be46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c40e0 con 0x7fef340b3f00 2023-10-16T08:18:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 0x7fef3414ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 0x7fef3414ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40604/0 (socket says 172.21.15.32:40604) 2023-10-16T08:18:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 -- 172.21.15.32:0/1440631915 learned_addr learned my addr 172.21.15.32:0/1440631915 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef393e1640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef340b3f00 0x7fef341548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef3a3e3640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef34159900 0x7fef34154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 -- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef340b3f00 msgr2=0x7fef341548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef340b3f00 0x7fef341548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 -- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef34159900 msgr2=0x7fef34154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef34159900 0x7fef34154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 -- 172.21.15.32:0/1440631915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30009420 con 0x7fef3406a5a0 2023-10-16T08:18:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef39be2640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 0x7fef3414ed80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fef300ae040 tx=0x7fef3001abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300b7450 con 0x7fef3406a5a0 2023-10-16T08:18:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef300b75d0 con 0x7fef3406a5a0 2023-10-16T08:18:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300b7980 con 0x7fef3406a5a0 2023-10-16T08:18:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef340c43c0 con 0x7fef3406a5a0 2023-10-16T08:18:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340c30f0 con 0x7fef3406a5a0 2023-10-16T08:18:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3404e170 con 0x7fef3406a5a0 2023-10-16T08:18:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fef300b7ae0 con 0x7fef3406a5a0 2023-10-16T08:18:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.705+0000 7fef2a7fc640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef180664a0 0x7fef18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.709+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fef3011e7c0 con 0x7fef3406a5a0 2023-10-16T08:18:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.709+0000 7fef393e1640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef180664a0 0x7fef18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.709+0000 7fef393e1640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef180664a0 0x7fef18068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fef340c9fc0 tx=0x7fef24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:10.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.709+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fef300ebb00 con 0x7fef3406a5a0 2023-10-16T08:18:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fef3406b380 con 0x7fef3406a5a0 2023-10-16T08:18:10.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef2a7fc640 1 -- 172.21.15.32:0/1440631915 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7fef300ef120 con 0x7fef3406a5a0 2023-10-16T08:18:10.875 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:10.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef180664a0 msgr2=0x7fef18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef180664a0 0x7fef18068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fef340c9fc0 tx=0x7fef24009040 comp rx=0 tx=0).stop 2023-10-16T08:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 msgr2=0x7fef3414ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 0x7fef3414ed80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fef300ae040 tx=0x7fef3001abb0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 shutdown_connections 2023-10-16T08:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef34159900 0x7fef34154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fef180664a0 0x7fef18068960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fef340b3f00 0x7fef341548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 --2- 172.21.15.32:0/1440631915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3406a5a0 0x7fef3414ed80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 >> 172.21.15.32:0/1440631915 conn(0x7fef340b0da0 msgr2=0x7fef340ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 shutdown_connections 2023-10-16T08:18:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:10.869+0000 7fef3be46640 1 -- 172.21.15.32:0/1440631915 wait complete. 2023-10-16T08:18:10.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:11.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.033+0000 7fed4bf3e640 1 Processor -- start 2023-10-16T08:18:11.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.033+0000 7fed4bf3e640 1 -- start start 2023-10-16T08:18:11.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4bf3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fed440cfd90 0x7fed4415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4bf3e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed440d0170 0x7fed440b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4bf3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed440b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4bf3e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4415e1a0 con 0x7fed440d0170 2023-10-16T08:18:11.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4bf3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4415e3a0 con 0x7fed440cfd90 2023-10-16T08:18:11.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4bf3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed440ce1c0 con 0x7fed440cdc00 2023-10-16T08:18:11.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed440d0170 0x7fed440b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed440d0170 0x7fed440b81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40606/0 (socket says 172.21.15.32:40606) 2023-10-16T08:18:11.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 -- 172.21.15.32:0/1720240575 learned_addr learned my addr 172.21.15.32:0/1720240575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:11.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed49cda640 1 --2- 172.21.15.32:0/1720240575 >> v2:172.21.15.186:3300/0 conn(0x7fed440cfd90 0x7fed4415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4a4db640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed440b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 -- 172.21.15.32:0/1720240575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 msgr2=0x7fed440b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed440b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 -- 172.21.15.32:0/1720240575 >> v2:172.21.15.186:3300/0 conn(0x7fed440cfd90 msgr2=0x7fed4415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 --2- 172.21.15.32:0/1720240575 >> v2:172.21.15.186:3300/0 conn(0x7fed440cfd90 0x7fed4415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed494d9640 1 -- 172.21.15.32:0/1720240575 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4415e640 con 0x7fed440d0170 2023-10-16T08:18:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.037+0000 7fed4a4db640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed440b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed494d9640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440d0170 0x7fed440b81b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fed440d0f00 tx=0x7fed3401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e3f946ea787839 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed48cd8640 1 -- 172.21.15.32:0/1720240575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340ab600 con 0x7fed440d0170 2023-10-16T08:18:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed48cd8640 1 -- 172.21.15.32:0/1720240575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed340a7050 con 0x7fed440d0170 2023-10-16T08:18:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed48cd8640 1 -- 172.21.15.32:0/1720240575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340ab980 con 0x7fed440d0170 2023-10-16T08:18:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- 172.21.15.32:0/1720240575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440d0170 msgr2=0x7fed440b81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440d0170 0x7fed440b81b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fed440d0f00 tx=0x7fed3401c500 comp rx=0 tx=0).stop 2023-10-16T08:18:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- 172.21.15.32:0/1720240575 shutdown_connections 2023-10-16T08:18:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed440b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/1720240575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440d0170 0x7fed440b81b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/1720240575 >> v2:172.21.15.186:3300/0 conn(0x7fed440cfd90 0x7fed4415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- 172.21.15.32:0/1720240575 >> 172.21.15.32:0/1720240575 conn(0x7fed440719c0 msgr2=0x7fed44071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- 172.21.15.32:0/1720240575 shutdown_connections 2023-10-16T08:18:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- 172.21.15.32:0/1720240575 wait complete. 2023-10-16T08:18:11.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 Processor -- start 2023-10-16T08:18:11.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- start start 2023-10-16T08:18:11.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed44148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 0x7fed4415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed440d0170 0x7fed440d2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed441569c0 con 0x7fed440cfd90 2023-10-16T08:18:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed44154cd0 con 0x7fed440d0170 2023-10-16T08:18:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.041+0000 7fed4bf3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed44154f70 con 0x7fed440cdc00 2023-10-16T08:18:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 0x7fed4415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 0x7fed4415b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40622/0 (socket says 172.21.15.32:40622) 2023-10-16T08:18:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 -- 172.21.15.32:0/784248671 learned_addr learned my addr 172.21.15.32:0/784248671 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed4a4db640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed440d0170 0x7fed440d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed49cda640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed44148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 -- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 msgr2=0x7fed44148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed44148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 -- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed440d0170 msgr2=0x7fed440d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed440d0170 0x7fed440d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 -- 172.21.15.32:0/784248671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed340094d0 con 0x7fed440cfd90 2023-10-16T08:18:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed4a4db640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed440d0170 0x7fed440d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:11.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed49cda640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed44148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed494d9640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 0x7fed4415b7c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed3401cda0 tx=0x7fed340192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340b5400 con 0x7fed440cfd90 2023-10-16T08:18:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed340b5560 con 0x7fed440cfd90 2023-10-16T08:18:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340b5910 con 0x7fed440cfd90 2023-10-16T08:18:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed44152040 con 0x7fed440cfd90 2023-10-16T08:18:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed44156590 con 0x7fed440cfd90 2023-10-16T08:18:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed44156b30 con 0x7fed440cfd90 2023-10-16T08:18:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.045+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed340b5a70 con 0x7fed440cfd90 2023-10-16T08:18:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.049+0000 7fed3a7fc640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed280664a0 0x7fed28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.049+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fed3411d750 con 0x7fed440cfd90 2023-10-16T08:18:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.049+0000 7fed49cda640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed280664a0 0x7fed28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.049+0000 7fed49cda640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed280664a0 0x7fed28068960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fed40009840 tx=0x7fed40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.053+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed340ea840 con 0x7fed440cfd90 2023-10-16T08:18:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.221+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fed4404e170 con 0x7fed440cfd90 2023-10-16T08:18:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.221+0000 7fed3a7fc640 1 -- 172.21.15.32:0/784248671 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fed340ede60 con 0x7fed440cfd90 2023-10-16T08:18:11.229 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed280664a0 msgr2=0x7fed28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed280664a0 0x7fed28068960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fed40009840 tx=0x7fed40008040 comp rx=0 tx=0).stop 2023-10-16T08:18:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 msgr2=0x7fed4415b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 0x7fed4415b7c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed3401cda0 tx=0x7fed340192a0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 shutdown_connections 2023-10-16T08:18:11.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed440d0170 0x7fed440d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed440cfd90 0x7fed4415b7c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed280664a0 0x7fed28068960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 --2- 172.21.15.32:0/784248671 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fed440cdc00 0x7fed44148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 >> 172.21.15.32:0/784248671 conn(0x7fed440719c0 msgr2=0x7fed44071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 shutdown_connections 2023-10-16T08:18:11.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.225+0000 7fed4bf3e640 1 -- 172.21.15.32:0/784248671 wait complete. 2023-10-16T08:18:11.250 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:11.251 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:11.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd879e09640 1 Processor -- start 2023-10-16T08:18:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd879e09640 1 -- start start 2023-10-16T08:18:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd879e09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8741597c0 0x7fd8740ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd879e09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8740723d0 con 0x7fd8740adc10 2023-10-16T08:18:11.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd879e09640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8740717c0 con 0x7fd874159ba0 2023-10-16T08:18:11.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd879e09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd874071aa0 con 0x7fd8741597c0 2023-10-16T08:18:11.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd8737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8741597c0 0x7fd8740ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd8737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8741597c0 0x7fd8740ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40638/0 (socket says 172.21.15.32:40638) 2023-10-16T08:18:11.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.437+0000 7fd8737fe640 1 -- 172.21.15.32:0/447641586 learned_addr learned my addr 172.21.15.32:0/447641586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:11.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd8737fe640 1 -- 172.21.15.32:0/447641586 >> v1:172.21.15.186:6789/0 conn(0x7fd874159ba0 legacy=0x7fd8740cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:11.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd8737fe640 1 -- 172.21.15.32:0/447641586 >> v1:172.21.15.32:6789/0 conn(0x7fd8740adc10 legacy=0x7fd87415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd8737fe640 1 -- 172.21.15.32:0/447641586 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd874071d80 con 0x7fd8741597c0 2023-10-16T08:18:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd872ffd640 1 -- 172.21.15.32:0/447641586 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512569396 0 0) 0x7fd8740723d0 con 0x7fd8740adc10 2023-10-16T08:18:11.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd8737fe640 1 --2- 172.21.15.32:0/447641586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd8740ad6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd864009870 tx=0x7fd86401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fe4e405d90ba14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd872ffd640 1 -- 172.21.15.32:0/447641586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640aa610 con 0x7fd8741597c0 2023-10-16T08:18:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd872ffd640 1 -- 172.21.15.32:0/447641586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8640a6040 con 0x7fd8741597c0 2023-10-16T08:18:11.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd872ffd640 1 -- 172.21.15.32:0/447641586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640aa990 con 0x7fd8741597c0 2023-10-16T08:18:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd879e09640 1 -- 172.21.15.32:0/447641586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 msgr2=0x7fd8740ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.441+0000 7fd879e09640 1 --2- 172.21.15.32:0/447641586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd8740ad6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd864009870 tx=0x7fd86401c510 comp rx=0 tx=0).stop 2023-10-16T08:18:11.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- 172.21.15.32:0/447641586 shutdown_connections 2023-10-16T08:18:11.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 --2- 172.21.15.32:0/447641586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd8740ad6d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- 172.21.15.32:0/447641586 >> 172.21.15.32:0/447641586 conn(0x7fd87406c190 msgr2=0x7fd87406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:11.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- 172.21.15.32:0/447641586 shutdown_connections 2023-10-16T08:18:11.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- 172.21.15.32:0/447641586 wait complete. 2023-10-16T08:18:11.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 Processor -- start 2023-10-16T08:18:11.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- start start 2023-10-16T08:18:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8740adc10 0x7fd874157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd87414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd874159ba0 0x7fd8741518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741529d0 con 0x7fd8741597c0 2023-10-16T08:18:11.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd874156fe0 con 0x7fd8740adc10 2023-10-16T08:18:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd879e09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd874152410 con 0x7fd874159ba0 2023-10-16T08:18:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd87414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd87414f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40640/0 (socket says 172.21.15.32:40640) 2023-10-16T08:18:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 -- 172.21.15.32:0/1710147731 learned_addr learned my addr 172.21.15.32:0/1710147731 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:11.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd8737fe640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8740adc10 0x7fd874157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd873fff640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd874159ba0 0x7fd8741518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 -- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd874159ba0 msgr2=0x7fd8741518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd874159ba0 0x7fd8741518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 -- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8740adc10 msgr2=0x7fd874157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8740adc10 0x7fd874157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 -- 172.21.15.32:0/1710147731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640094d0 con 0x7fd8741597c0 2023-10-16T08:18:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.445+0000 7fd86bfff640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd87414f0b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c3c0 tx=0x7fd85c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0052d0 con 0x7fd8741597c0 2023-10-16T08:18:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c005430 con 0x7fd8741597c0 2023-10-16T08:18:11.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c003840 con 0x7fd8741597c0 2023-10-16T08:18:11.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8741526f0 con 0x7fd8741597c0 2023-10-16T08:18:11.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87414fbb0 con 0x7fd8741597c0 2023-10-16T08:18:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874156ad0 con 0x7fd8741597c0 2023-10-16T08:18:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd85c0039a0 con 0x7fd8741597c0 2023-10-16T08:18:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd870ff9640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8540664d0 0x7fd854068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fd85c083650 con 0x7fd8741597c0 2023-10-16T08:18:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.449+0000 7fd8737fe640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8540664d0 0x7fd854068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.453+0000 7fd8737fe640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8540664d0 0x7fd854068990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8640ae000 tx=0x7fd8640b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.453+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd874156ad0 con 0x7fd8741597c0 2023-10-16T08:18:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd874156ad0 con 0x7fd8741597c0 2023-10-16T08:18:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd870ff9640 1 -- 172.21.15.32:0/1710147731 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7fd874156ad0 con 0x7fd8741597c0 2023-10-16T08:18:11.619 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:11.620 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8540664d0 msgr2=0x7fd854068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8540664d0 0x7fd854068990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8640ae000 tx=0x7fd8640b5040 comp rx=0 tx=0).stop 2023-10-16T08:18:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 msgr2=0x7fd87414f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd87414f0b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c3c0 tx=0x7fd85c00c880 comp rx=0 tx=0).stop 2023-10-16T08:18:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 shutdown_connections 2023-10-16T08:18:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8540664d0 0x7fd854068990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd874159ba0 0x7fd8741518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8741597c0 0x7fd87414f0b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 --2- 172.21.15.32:0/1710147731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8740adc10 0x7fd874157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 >> 172.21.15.32:0/1710147731 conn(0x7fd87406c190 msgr2=0x7fd8740af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 shutdown_connections 2023-10-16T08:18:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.613+0000 7fd879e09640 1 -- 172.21.15.32:0/1710147731 wait complete. 2023-10-16T08:18:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:11.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 Processor -- start 2023-10-16T08:18:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 -- start start 2023-10-16T08:18:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffad00c9f20 0x7ffad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad00b4e60 con 0x7ffad00715c0 2023-10-16T08:18:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad00b1fc0 con 0x7ffad00c9f20 2023-10-16T08:18:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.781+0000 7ffad77d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015e680 con 0x7ffad015a940 2023-10-16T08:18:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5575640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5d76640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ffad00c9f20 0x7ffad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad4d74640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7ffad00715c0 0x7ffad00c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:48476/0 (socket says 172.21.15.32:48476) 2023-10-16T08:18:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5575640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:41290/0 (socket says 172.21.15.32:41290) 2023-10-16T08:18:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad4d74640 1 -- 172.21.15.32:0/4168245992 learned_addr learned my addr 172.21.15.32:0/4168245992 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5575640 1 -- 172.21.15.32:0/4168245992 >> v2:172.21.15.186:3300/0 conn(0x7ffad00c9f20 msgr2=0x7ffad006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5575640 1 --2- 172.21.15.32:0/4168245992 >> v2:172.21.15.186:3300/0 conn(0x7ffad00c9f20 0x7ffad006e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5575640 1 -- 172.21.15.32:0/4168245992 >> v1:172.21.15.186:6789/0 conn(0x7ffad00715c0 legacy=0x7ffad00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5575640 1 -- 172.21.15.32:0/4168245992 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad015e960 con 0x7ffad015a940 2023-10-16T08:18:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffad5d76640 1 --2- 172.21.15.32:0/4168245992 >> v2:172.21.15.186:3300/0 conn(0x7ffad00c9f20 0x7ffad006e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.785+0000 7ffac7fff640 1 -- 172.21.15.32:0/4168245992 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037981133 0 0) 0x7ffad00b4e60 con 0x7ffad00715c0 2023-10-16T08:18:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad5575640 1 --2- 172.21.15.32:0/4168245992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00b4850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0097c0 tx=0x7ffacc01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12055d79f49cfda server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffac7fff640 1 -- 172.21.15.32:0/4168245992 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0a8970 con 0x7ffad015a940 2023-10-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffac7fff640 1 -- 172.21.15.32:0/4168245992 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc0a8ad0 con 0x7ffad015a940 2023-10-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffac7fff640 1 -- 172.21.15.32:0/4168245992 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0a8e60 con 0x7ffad015a940 2023-10-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 -- 172.21.15.32:0/4168245992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 msgr2=0x7ffad00b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 --2- 172.21.15.32:0/4168245992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00b4850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0097c0 tx=0x7ffacc01ec70 comp rx=0 tx=0).stop 2023-10-16T08:18:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 -- 172.21.15.32:0/4168245992 shutdown_connections 2023-10-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 --2- 172.21.15.32:0/4168245992 >> v2:172.21.15.186:3300/0 conn(0x7ffad00c9f20 0x7ffad006e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 --2- 172.21.15.32:0/4168245992 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00b4850 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 -- 172.21.15.32:0/4168245992 >> 172.21.15.32:0/4168245992 conn(0x7ffad00cef20 msgr2=0x7ffad00cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 -- 172.21.15.32:0/4168245992 shutdown_connections 2023-10-16T08:18:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 -- 172.21.15.32:0/4168245992 wait complete. 2023-10-16T08:18:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 Processor -- start 2023-10-16T08:18:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.789+0000 7ffad77d9640 1 -- start start 2023-10-16T08:18:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00715c0 0x7ffad00adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 0x7ffad0153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00ae1e0 con 0x7ffad00c9f20 2023-10-16T08:18:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015c780 con 0x7ffad00715c0 2023-10-16T08:18:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015ca60 con 0x7ffad015a940 2023-10-16T08:18:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 0x7ffad0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 0x7ffad0153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40654/0 (socket says 172.21.15.32:40654) 2023-10-16T08:18:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 -- 172.21.15.32:0/3138885257 learned_addr learned my addr 172.21.15.32:0/3138885257 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad5575640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00715c0 0x7ffad00adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad5d76640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 -- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 msgr2=0x7ffad00ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 -- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00715c0 msgr2=0x7ffad00adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00715c0 0x7ffad00adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 -- 172.21.15.32:0/3138885257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8009a70 con 0x7ffad00c9f20 2023-10-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad5d76640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad5575640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00715c0 0x7ffad00adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad4d74640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 0x7ffad0153380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffac00011f0 tx=0x7ffac000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00074e0 con 0x7ffad00c9f20 2023-10-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0007640 con 0x7ffad00c9f20 2023-10-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004a10 con 0x7ffad00c9f20 2023-10-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc009420 con 0x7ffad00c9f20 2023-10-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad015d020 con 0x7ffad00c9f20 2023-10-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad004e170 con 0x7ffad00c9f20 2023-10-16T08:18:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.793+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffac0004b70 con 0x7ffad00c9f20 2023-10-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.797+0000 7ffac5ffb640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ffab40664d0 0x7ffab4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.797+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7ffac0083240 con 0x7ffad00c9f20 2023-10-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.797+0000 7ffad5575640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ffab40664d0 0x7ffab4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.797+0000 7ffad5575640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ffab40664d0 0x7ffab4068990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0b2000 tx=0x7ffacc0aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.801+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffac0050470 con 0x7ffad00c9f20 2023-10-16T08:18:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.957+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffad00ae320 con 0x7ffad00c9f20 2023-10-16T08:18:11.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.957+0000 7ffac5ffb640 1 -- 172.21.15.32:0/3138885257 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffac0053a90 con 0x7ffad00c9f20 2023-10-16T08:18:11.965 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:11.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ffab40664d0 msgr2=0x7ffab4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ffab40664d0 0x7ffab4068990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0b2000 tx=0x7ffacc0aa070 comp rx=0 tx=0).stop 2023-10-16T08:18:11.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 msgr2=0x7ffad0153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:11.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 0x7ffad0153380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffac00011f0 tx=0x7ffac000c9b0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 shutdown_connections 2023-10-16T08:18:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ffab40664d0 0x7ffab4068990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffad015a940 0x7ffad00ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffad00c9f20 0x7ffad0153380 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 --2- 172.21.15.32:0/3138885257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00715c0 0x7ffad00adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 >> 172.21.15.32:0/3138885257 conn(0x7ffad00cef20 msgr2=0x7ffad00b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 shutdown_connections 2023-10-16T08:18:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:11.961+0000 7ffad77d9640 1 -- 172.21.15.32:0/3138885257 wait complete. 2023-10-16T08:18:11.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.125+0000 7fe027b3e640 1 Processor -- start 2023-10-16T08:18:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 -- start start 2023-10-16T08:18:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe02015cc60 0x7fe0200cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe020161260 0x7fe0200ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0200cebd0 0x7fe0200b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200cf020 con 0x7fe02015cc60 2023-10-16T08:18:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200cc330 con 0x7fe020161260 2023-10-16T08:18:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe027b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200cb720 con 0x7fe0200cebd0 2023-10-16T08:18:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe02015cc60 0x7fe0200cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe02015cc60 0x7fe0200cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40656/0 (socket says 172.21.15.32:40656) 2023-10-16T08:18:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 -- 172.21.15.32:0/3223871014 learned_addr learned my addr 172.21.15.32:0/3223871014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0260db640 1 --2- 172.21.15.32:0/3223871014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0200cebd0 0x7fe0200b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0250d9640 1 --2- 172.21.15.32:0/3223871014 >> v2:172.21.15.186:3300/0 conn(0x7fe020161260 0x7fe0200ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 -- 172.21.15.32:0/3223871014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0200cebd0 msgr2=0x7fe0200b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 --2- 172.21.15.32:0/3223871014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0200cebd0 0x7fe0200b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 -- 172.21.15.32:0/3223871014 >> v2:172.21.15.186:3300/0 conn(0x7fe020161260 msgr2=0x7fe0200ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:12.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 --2- 172.21.15.32:0/3223871014 >> v2:172.21.15.186:3300/0 conn(0x7fe020161260 0x7fe0200ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.129+0000 7fe0258da640 1 -- 172.21.15.32:0/3223871014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200cba00 con 0x7fe02015cc60 2023-10-16T08:18:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe0258da640 1 --2- 172.21.15.32:0/3223871014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02015cc60 0x7fe0200cb040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe0200cdf60 tx=0x7fe01c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807139961134b6a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe0248d8640 1 -- 172.21.15.32:0/3223871014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0ab600 con 0x7fe02015cc60 2023-10-16T08:18:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe0248d8640 1 -- 172.21.15.32:0/3223871014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c0a7050 con 0x7fe02015cc60 2023-10-16T08:18:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe0248d8640 1 -- 172.21.15.32:0/3223871014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0ab980 con 0x7fe02015cc60 2023-10-16T08:18:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 -- 172.21.15.32:0/3223871014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02015cc60 msgr2=0x7fe0200cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 --2- 172.21.15.32:0/3223871014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02015cc60 0x7fe0200cb040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe0200cdf60 tx=0x7fe01c01c210 comp rx=0 tx=0).stop 2023-10-16T08:18:12.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 -- 172.21.15.32:0/3223871014 shutdown_connections 2023-10-16T08:18:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 --2- 172.21.15.32:0/3223871014 >> v2:172.21.15.186:3300/0 conn(0x7fe020161260 0x7fe0200ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 --2- 172.21.15.32:0/3223871014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe0200cebd0 0x7fe0200b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 --2- 172.21.15.32:0/3223871014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02015cc60 0x7fe0200cb040 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 -- 172.21.15.32:0/3223871014 >> 172.21.15.32:0/3223871014 conn(0x7fe02006e570 msgr2=0x7fe02006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 -- 172.21.15.32:0/3223871014 shutdown_connections 2023-10-16T08:18:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 -- 172.21.15.32:0/3223871014 wait complete. 2023-10-16T08:18:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 Processor -- start 2023-10-16T08:18:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.133+0000 7fe027b3e640 1 -- start start 2023-10-16T08:18:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 0x7fe02014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe02015cc60 0x7fe0200b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020161260 0x7fe0200b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02014e960 con 0x7fe0200cebd0 2023-10-16T08:18:12.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02014d3d0 con 0x7fe020161260 2023-10-16T08:18:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02014e4d0 con 0x7fe02015cc60 2023-10-16T08:18:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 0x7fe02014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 0x7fe02014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40670/0 (socket says 172.21.15.32:40670) 2023-10-16T08:18:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 -- 172.21.15.32:0/2012277932 learned_addr learned my addr 172.21.15.32:0/2012277932 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0250d9640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe02015cc60 0x7fe0200b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0260db640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020161260 0x7fe0200b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 -- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe02015cc60 msgr2=0x7fe0200b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe02015cc60 0x7fe0200b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 -- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020161260 msgr2=0x7fe0200b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020161260 0x7fe0200b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 -- 172.21.15.32:0/2012277932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0094d0 con 0x7fe0200cebd0 2023-10-16T08:18:12.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0250d9640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe02015cc60 0x7fe0200b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:12.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0260db640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020161260 0x7fe0200b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:12.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0258da640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 0x7fe02014ce90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ae040 tx=0x7fe01c01aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0b6620 con 0x7fe0200cebd0 2023-10-16T08:18:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c0b6780 con 0x7fe0200cebd0 2023-10-16T08:18:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0b6b10 con 0x7fe0200cebd0 2023-10-16T08:18:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02014d8c0 con 0x7fe0200cebd0 2023-10-16T08:18:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02014de80 con 0x7fe0200cebd0 2023-10-16T08:18:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02004e170 con 0x7fe0200cebd0 2023-10-16T08:18:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.137+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe01c0b2040 con 0x7fe0200cebd0 2023-10-16T08:18:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.141+0000 7fe0167fc640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe0040664a0 0x7fe004068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.141+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fe01c11e4f0 con 0x7fe0200cebd0 2023-10-16T08:18:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.141+0000 7fe0250d9640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe0040664a0 0x7fe004068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.141+0000 7fe0250d9640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe0040664a0 0x7fe004068960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe0201f86f0 tx=0x7fe010008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.145+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe01c0eb670 con 0x7fe0200cebd0 2023-10-16T08:18:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0200b6830 con 0x7fe0200cebd0 2023-10-16T08:18:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe0167fc640 1 -- 172.21.15.32:0/2012277932 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7fe01c0eec90 con 0x7fe0200cebd0 2023-10-16T08:18:12.308 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:12.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe0040664a0 msgr2=0x7fe004068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe0040664a0 0x7fe004068960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe0201f86f0 tx=0x7fe010008040 comp rx=0 tx=0).stop 2023-10-16T08:18:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 msgr2=0x7fe02014ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 0x7fe02014ce90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ae040 tx=0x7fe01c01aa50 comp rx=0 tx=0).stop 2023-10-16T08:18:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 shutdown_connections 2023-10-16T08:18:12.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe020161260 0x7fe0200b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe0040664a0 0x7fe004068960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe02015cc60 0x7fe0200b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.301+0000 7fe027b3e640 1 --2- 172.21.15.32:0/2012277932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0200cebd0 0x7fe02014ce90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.305+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 >> 172.21.15.32:0/2012277932 conn(0x7fe02006e570 msgr2=0x7fe02006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.305+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 shutdown_connections 2023-10-16T08:18:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.305+0000 7fe027b3e640 1 -- 172.21.15.32:0/2012277932 wait complete. 2023-10-16T08:18:12.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:12.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7dacf0f640 1 Processor -- start 2023-10-16T08:18:12.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7dacf0f640 1 -- start start 2023-10-16T08:18:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7dacf0f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7da80ceda0 0x7f7da8167be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7dacf0f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80cc330 con 0x7f7da80cb040 2023-10-16T08:18:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7dacf0f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80cb4d0 con 0x7f7da815cc60 2023-10-16T08:18:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7dacf0f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da80cb720 con 0x7f7da80ceda0 2023-10-16T08:18:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7da80ceda0 0x7f7da8167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7da80ceda0 0x7f7da8167be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40684/0 (socket says 172.21.15.32:40684) 2023-10-16T08:18:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7d9dd9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7da80cb040 0x7f7da80ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59226/0 (socket says 172.21.15.32:59226) 2023-10-16T08:18:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da6d9d640 1 -- 172.21.15.32:0/3238017437 learned_addr learned my addr 172.21.15.32:0/3238017437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da6d9d640 1 -- 172.21.15.32:0/3238017437 >> v1:172.21.15.186:6789/0 conn(0x7f7da815cc60 legacy=0x7f7da8161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da6d9d640 1 -- 172.21.15.32:0/3238017437 >> v1:172.21.15.32:6789/0 conn(0x7f7da80cb040 legacy=0x7f7da80ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da6d9d640 1 -- 172.21.15.32:0/3238017437 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80cba00 con 0x7f7da80ceda0 2023-10-16T08:18:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.469+0000 7f7da5d9b640 1 -- 172.21.15.32:0/3238017437 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439756346 0 0) 0x7f7da80cc330 con 0x7f7da80cb040 2023-10-16T08:18:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7da6d9d640 1 --2- 172.21.15.32:0/3238017437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80ceda0 0x7f7da8167be0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009e50 tx=0x7f7da001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc626b9351890c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7da5d9b640 1 -- 172.21.15.32:0/3238017437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00a58a0 con 0x7f7da80ceda0 2023-10-16T08:18:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7da5d9b640 1 -- 172.21.15.32:0/3238017437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00a5a00 con 0x7f7da80ceda0 2023-10-16T08:18:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7da5d9b640 1 -- 172.21.15.32:0/3238017437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00a5d90 con 0x7f7da80ceda0 2023-10-16T08:18:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3238017437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80ceda0 msgr2=0x7f7da8167be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3238017437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80ceda0 0x7f7da8167be0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009e50 tx=0x7f7da001c9e0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3238017437 shutdown_connections 2023-10-16T08:18:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3238017437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80ceda0 0x7f7da8167be0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3238017437 >> 172.21.15.32:0/3238017437 conn(0x7f7da806e570 msgr2=0x7f7da806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3238017437 shutdown_connections 2023-10-16T08:18:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.473+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3238017437 wait complete. 2023-10-16T08:18:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 Processor -- start 2023-10-16T08:18:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- start start 2023-10-16T08:18:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 0x7f7da80c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80ceda0 0x7f7da8154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da815cc60 0x7f7da8154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da8154e90 con 0x7f7da80cb040 2023-10-16T08:18:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b2bd0 con 0x7f7da815cc60 2023-10-16T08:18:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b4870 con 0x7f7da80ceda0 2023-10-16T08:18:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 0x7f7da80c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 0x7f7da80c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40688/0 (socket says 172.21.15.32:40688) 2023-10-16T08:18:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 -- 172.21.15.32:0/3741953937 learned_addr learned my addr 172.21.15.32:0/3741953937 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da6d9d640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da815cc60 0x7f7da8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7d9dd9b640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80ceda0 0x7f7da8154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 -- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80ceda0 msgr2=0x7f7da8154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80ceda0 0x7f7da8154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 -- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da815cc60 msgr2=0x7f7da8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da815cc60 0x7f7da8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 -- 172.21.15.32:0/3741953937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0009b30 con 0x7f7da80cb040 2023-10-16T08:18:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7d9dd9b640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80ceda0 0x7f7da8154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7da659c640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 0x7f7da80c8360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c3c0 tx=0x7f7d9800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980052d0 con 0x7f7da80cb040 2023-10-16T08:18:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98005430 con 0x7f7da80cb040 2023-10-16T08:18:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98003840 con 0x7f7da80cb040 2023-10-16T08:18:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da80b4b50 con 0x7f7da80cb040 2023-10-16T08:18:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da8155190 con 0x7f7da80cb040 2023-10-16T08:18:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.477+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da80b46d0 con 0x7f7da80cb040 2023-10-16T08:18:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.481+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d980039a0 con 0x7f7da80cb040 2023-10-16T08:18:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.481+0000 7f7d9f7fe640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7d880664a0 0x7f7d88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.481+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f7d980836e0 con 0x7f7da80cb040 2023-10-16T08:18:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.481+0000 7f7d9dd9b640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7d880664a0 0x7f7d88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.481+0000 7f7d9dd9b640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7d880664a0 0x7f7d88068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7da8159b80 tx=0x7f7d90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.485+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7da80b46d0 con 0x7f7da80cb040 2023-10-16T08:18:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.637+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f7da80b46d0 con 0x7f7da80cb040 2023-10-16T08:18:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7d9f7fe640 1 -- 172.21.15.32:0/3741953937 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7f7da80b46d0 con 0x7f7da80cb040 2023-10-16T08:18:12.646 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:12.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7d880664a0 msgr2=0x7f7d88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7d880664a0 0x7f7d88068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7da8159b80 tx=0x7f7d90009040 comp rx=0 tx=0).stop 2023-10-16T08:18:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 msgr2=0x7f7da80c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 0x7f7da80c8360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c3c0 tx=0x7f7d9800c880 comp rx=0 tx=0).stop 2023-10-16T08:18:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 shutdown_connections 2023-10-16T08:18:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7da815cc60 0x7f7da8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7d880664a0 0x7f7d88068960 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7da80ceda0 0x7f7da8154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 --2- 172.21.15.32:0/3741953937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7da80cb040 0x7f7da80c8360 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 >> 172.21.15.32:0/3741953937 conn(0x7f7da806e570 msgr2=0x7f7da815f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 shutdown_connections 2023-10-16T08:18:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.641+0000 7f7dacf0f640 1 -- 172.21.15.32:0/3741953937 wait complete. 2023-10-16T08:18:12.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.805+0000 7feab6254640 1 Processor -- start 2023-10-16T08:18:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.805+0000 7feab6254640 1 -- start start 2023-10-16T08:18:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feab6254640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 0x7feab006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feab6254640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feab00b3f00 0x7feab0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feab6254640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab006a3d0 0x7feab00cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feab6254640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab00cd740 con 0x7feab00b3f00 2023-10-16T08:18:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feab6254640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab006a840 con 0x7feab0159900 2023-10-16T08:18:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feab6254640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab00b4390 con 0x7feab006a3d0 2023-10-16T08:18:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaeffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feab00b3f00 0x7feab0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaf7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 0x7feab006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab006a3d0 0x7feab00cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaf7fe640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 0x7feab006e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:37644/0 (socket says 172.21.15.32:37644) 2023-10-16T08:18:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaf7fe640 1 -- 172.21.15.32:0/361537490 learned_addr learned my addr 172.21.15.32:0/361537490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaeffd640 1 -- 172.21.15.32:0/361537490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab006a3d0 msgr2=0x7feab00cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaeffd640 1 --2- 172.21.15.32:0/361537490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab006a3d0 0x7feab00cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaeffd640 1 -- 172.21.15.32:0/361537490 >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 msgr2=0x7feab006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaeffd640 1 --2- 172.21.15.32:0/361537490 >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 0x7feab006e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaeffd640 1 -- 172.21.15.32:0/361537490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab00b4600 con 0x7feab00b3f00 2023-10-16T08:18:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaf7fe640 1 --2- 172.21.15.32:0/361537490 >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 0x7feab006e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:12.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.809+0000 7feaaffff640 1 --2- 172.21.15.32:0/361537490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab006a3d0 0x7feab00cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:12.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feaaeffd640 1 --2- 172.21.15.32:0/361537490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab00b3f00 0x7feab0069e90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feab00b4530 tx=0x7feaa401c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79ebfd350107dc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feaae7fc640 1 -- 172.21.15.32:0/361537490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa40ab600 con 0x7feab00b3f00 2023-10-16T08:18:12.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feaae7fc640 1 -- 172.21.15.32:0/361537490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa40a7050 con 0x7feab00b3f00 2023-10-16T08:18:12.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feaae7fc640 1 -- 172.21.15.32:0/361537490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa40ab980 con 0x7feab00b3f00 2023-10-16T08:18:12.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 -- 172.21.15.32:0/361537490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab00b3f00 msgr2=0x7feab0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 --2- 172.21.15.32:0/361537490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab00b3f00 0x7feab0069e90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feab00b4530 tx=0x7feaa401c150 comp rx=0 tx=0).stop 2023-10-16T08:18:12.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 -- 172.21.15.32:0/361537490 shutdown_connections 2023-10-16T08:18:12.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 --2- 172.21.15.32:0/361537490 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab006a3d0 0x7feab00cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 --2- 172.21.15.32:0/361537490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab00b3f00 0x7feab0069e90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 --2- 172.21.15.32:0/361537490 >> v2:172.21.15.186:3300/0 conn(0x7feab0159900 0x7feab006e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 -- 172.21.15.32:0/361537490 >> 172.21.15.32:0/361537490 conn(0x7feab00b0da0 msgr2=0x7feab00ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 -- 172.21.15.32:0/361537490 shutdown_connections 2023-10-16T08:18:12.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.813+0000 7feab6254640 1 -- 172.21.15.32:0/361537490 wait complete. 2023-10-16T08:18:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 Processor -- start 2023-10-16T08:18:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 -- start start 2023-10-16T08:18:12.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 0x7feab00c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab00b3f00 0x7feab00c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab0159900 0x7feab00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab015ba20 con 0x7feab006a3d0 2023-10-16T08:18:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab00c41f0 con 0x7feab00b3f00 2023-10-16T08:18:12.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feab6254640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab01f8830 con 0x7feab0159900 2023-10-16T08:18:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 0x7feab00c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 0x7feab00c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40694/0 (socket says 172.21.15.32:40694) 2023-10-16T08:18:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 -- 172.21.15.32:0/3093573430 learned_addr learned my addr 172.21.15.32:0/3093573430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaeffd640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab00b3f00 0x7feab00c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaffff640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab0159900 0x7feab00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 -- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab0159900 msgr2=0x7feab00ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab0159900 0x7feab00ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 -- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab00b3f00 msgr2=0x7feab00c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab00b3f00 0x7feab00c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 -- 172.21.15.32:0/3093573430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa8009a70 con 0x7feab006a3d0 2023-10-16T08:18:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7feaaf7fe640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 0x7feab00c3cb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e9d0 tx=0x7feaa000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000ca00 con 0x7feab006a3d0 2023-10-16T08:18:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa000cb60 con 0x7feab006a3d0 2023-10-16T08:18:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.817+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0013470 con 0x7feab006a3d0 2023-10-16T08:18:12.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa00094d0 con 0x7feab006a3d0 2023-10-16T08:18:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa4009420 con 0x7feab006a3d0 2023-10-16T08:18:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab004e170 con 0x7feab006a3d0 2023-10-16T08:18:12.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feaa00135d0 con 0x7feab006a3d0 2023-10-16T08:18:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7fea97fff640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fea9c0664a0 0x7fea9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7feaa00843d0 con 0x7feab006a3d0 2023-10-16T08:18:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.821+0000 7feaaeffd640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fea9c0664a0 0x7fea9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:12.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.825+0000 7feaaeffd640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fea9c0664a0 0x7fea9c068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feaa40ad000 tx=0x7feaa4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.825+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feaa0051570 con 0x7feab006a3d0 2023-10-16T08:18:12.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.981+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7feab006a0a0 con 0x7feab006a3d0 2023-10-16T08:18:12.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.981+0000 7fea97fff640 1 -- 172.21.15.32:0/3093573430 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7feaa0054b90 con 0x7feab006a3d0 2023-10-16T08:18:12.989 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fea9c0664a0 msgr2=0x7fea9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fea9c0664a0 0x7fea9c068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feaa40ad000 tx=0x7feaa4008040 comp rx=0 tx=0).stop 2023-10-16T08:18:12.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 msgr2=0x7feab00c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:12.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 0x7feab00c3cb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e9d0 tx=0x7feaa000edd0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 shutdown_connections 2023-10-16T08:18:12.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fea9c0664a0 0x7fea9c068960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7feab0159900 0x7feab00ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab00b3f00 0x7feab00c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 --2- 172.21.15.32:0/3093573430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feab006a3d0 0x7feab00c3cb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 >> 172.21.15.32:0/3093573430 conn(0x7feab00b0da0 msgr2=0x7feab00ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 shutdown_connections 2023-10-16T08:18:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:12.985+0000 7feab6254640 1 -- 172.21.15.32:0/3093573430 wait complete. 2023-10-16T08:18:13.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:13.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 Processor -- start 2023-10-16T08:18:13.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 -- start start 2023-10-16T08:18:13.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e0159ba0 0x7f57e00cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:13.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57e00c8cb0 0x7f57e00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:13.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00723d0 con 0x7f57e01597c0 2023-10-16T08:18:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e00717c0 con 0x7f57e00c8cb0 2023-10-16T08:18:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57e5a68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0071aa0 con 0x7f57e0159ba0 2023-10-16T08:18:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57e00c8cb0 0x7f57e00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57e00c8cb0 0x7f57e00cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37854/0 (socket says 172.21.15.32:37854) 2023-10-16T08:18:13.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 -- 172.21.15.32:0/1907324328 learned_addr learned my addr 172.21.15.32:0/1907324328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:13.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57deffd640 1 --2- 172.21.15.32:0/1907324328 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e0159ba0 0x7f57e00cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:13.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 -- 172.21.15.32:0/1907324328 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e0159ba0 msgr2=0x7f57e00cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 --2- 172.21.15.32:0/1907324328 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e0159ba0 0x7f57e00cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 -- 172.21.15.32:0/1907324328 >> v1:172.21.15.186:6789/0 conn(0x7f57e01597c0 legacy=0x7f57e00ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.149+0000 7f57dffff640 1 -- 172.21.15.32:0/1907324328 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0071d80 con 0x7f57e00c8cb0 2023-10-16T08:18:13.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57dffff640 1 --2- 172.21.15.32:0/1907324328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e00c8cb0 0x7f57e00cb350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009e50 tx=0x7f57d801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c61e0b94a55062 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:13.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57de7fc640 1 -- 172.21.15.32:0/1907324328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80a57c0 con 0x7f57e00c8cb0 2023-10-16T08:18:13.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57de7fc640 1 -- 172.21.15.32:0/1907324328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d80a5920 con 0x7f57e00c8cb0 2023-10-16T08:18:13.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57de7fc640 1 -- 172.21.15.32:0/1907324328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80a5cb0 con 0x7f57e00c8cb0 2023-10-16T08:18:13.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 -- 172.21.15.32:0/1907324328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e00c8cb0 msgr2=0x7f57e00cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:13.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 --2- 172.21.15.32:0/1907324328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e00c8cb0 0x7f57e00cb350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009e50 tx=0x7f57d801c9a0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 -- 172.21.15.32:0/1907324328 shutdown_connections 2023-10-16T08:18:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 --2- 172.21.15.32:0/1907324328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e00c8cb0 0x7f57e00cb350 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 --2- 172.21.15.32:0/1907324328 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e0159ba0 0x7f57e00cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 -- 172.21.15.32:0/1907324328 >> 172.21.15.32:0/1907324328 conn(0x7f57e006c190 msgr2=0x7f57e006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 -- 172.21.15.32:0/1907324328 shutdown_connections 2023-10-16T08:18:13.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 -- 172.21.15.32:0/1907324328 wait complete. 2023-10-16T08:18:13.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.153+0000 7f57e5a68640 1 Processor -- start 2023-10-16T08:18:13.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- start start 2023-10-16T08:18:13.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e00c8cb0 0x7f57e015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 0x7f57e01f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0159ba0 0x7f57e01f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ac230 con 0x7f57e01597c0 2023-10-16T08:18:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e014f280 con 0x7f57e0159ba0 2023-10-16T08:18:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ac4d0 con 0x7f57e00c8cb0 2023-10-16T08:18:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 0x7f57e01f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 0x7f57e01f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37866/0 (socket says 172.21.15.32:37866) 2023-10-16T08:18:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 -- 172.21.15.32:0/3411100259 learned_addr learned my addr 172.21.15.32:0/3411100259 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57dffff640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0159ba0 0x7f57e01f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57df7fe640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e00c8cb0 0x7f57e015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 -- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e00c8cb0 msgr2=0x7f57e015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e00c8cb0 0x7f57e015c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 -- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0159ba0 msgr2=0x7f57e01f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0159ba0 0x7f57e01f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 -- 172.21.15.32:0/3411100259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d8009b30 con 0x7f57e01597c0 2023-10-16T08:18:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57df7fe640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e00c8cb0 0x7f57e015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57deffd640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 0x7f57e01f5c10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d5f0 tx=0x7f57d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40125f0 con 0x7f57e01597c0 2023-10-16T08:18:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4012750 con 0x7f57e01597c0 2023-10-16T08:18:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4012ae0 con 0x7f57e01597c0 2023-10-16T08:18:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e00ac7b0 con 0x7f57e01597c0 2023-10-16T08:18:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e014e770 con 0x7f57e01597c0 2023-10-16T08:18:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.157+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e0074010 con 0x7f57e01597c0 2023-10-16T08:18:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.161+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57d4012c40 con 0x7f57e01597c0 2023-10-16T08:18:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.161+0000 7f57c7fff640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57cc0664a0 0x7f57cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.161+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f57d40845a0 con 0x7f57e01597c0 2023-10-16T08:18:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.161+0000 7f57df7fe640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57cc0664a0 0x7f57cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.161+0000 7f57df7fe640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57cc0664a0 0x7f57cc068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f57d0003ec0 tx=0x7f57d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.165+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f57d4051730 con 0x7f57e01597c0 2023-10-16T08:18:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f57e00ad760 con 0x7f57e01597c0 2023-10-16T08:18:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57c7fff640 1 -- 172.21.15.32:0/3411100259 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f57d4054d50 con 0x7f57e01597c0 2023-10-16T08:18:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57cc0664a0 msgr2=0x7f57cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57cc0664a0 0x7f57cc068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f57d0003ec0 tx=0x7f57d0009040 comp rx=0 tx=0).stop 2023-10-16T08:18:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 msgr2=0x7f57e01f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 0x7f57e01f5c10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d5f0 tx=0x7f57d400dab0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 shutdown_connections 2023-10-16T08:18:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57e0159ba0 0x7f57e01f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57e01597c0 0x7f57e01f5c10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57cc0664a0 0x7f57cc068960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 --2- 172.21.15.32:0/3411100259 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57e00c8cb0 0x7f57e015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 >> 172.21.15.32:0/3411100259 conn(0x7f57e006c190 msgr2=0x7f57e014f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 shutdown_connections 2023-10-16T08:18:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:13.333+0000 7f57e5a68640 1 -- 172.21.15.32:0/3411100259 wait complete. 2023-10-16T08:18:13.359 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:18:13.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:13.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:13.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48813]: fuse finished with error 0 and tester_r 0 2023-10-16T08:18:13.382 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:17.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:18:17.279 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:18:19.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:18:19.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:19.485 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:18:19.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=testcephfs1_data,allow rw pool=data_new' 2023-10-16T08:18:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.689+0000 7f83d9246640 1 Processor -- start 2023-10-16T08:18:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.689+0000 7f83d9246640 1 -- start start 2023-10-16T08:18:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d9246640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f83d415cb60 0x7f83d4161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d9246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 0x7f83d40cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d9246640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d40cb200 con 0x7f83d40cc1e0 2023-10-16T08:18:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d9246640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d40cda90 con 0x7f83d415cb60 2023-10-16T08:18:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d9246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cb4a0 con 0x7f83d40cd240 2023-10-16T08:18:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83d40cc1e0 0x7f83d40ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33994/0 (socket says 172.21.15.32:33994) 2023-10-16T08:18:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 -- 172.21.15.32:0/1555496210 learned_addr learned my addr 172.21.15.32:0/1555496210 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83cbfff640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 0x7f83d40cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d2d9d640 1 --2- 172.21.15.32:0/1555496210 >> v2:172.21.15.186:3300/0 conn(0x7f83d415cb60 0x7f83d4161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422955206 0 0) 0x7f83d40cb200 con 0x7f83d40cc1e0 2023-10-16T08:18:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c0003530 con 0x7f83d40cc1e0 2023-10-16T08:18:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2647284863 0 0) 0x7f83c0003530 con 0x7f83d40cc1e0 2023-10-16T08:18:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 msgr2=0x7f83d40cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 0x7f83d40cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 >> v2:172.21.15.186:3300/0 conn(0x7f83d415cb60 msgr2=0x7f83d4161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 --2- 172.21.15.32:0/1555496210 >> v2:172.21.15.186:3300/0 conn(0x7f83d415cb60 0x7f83d4161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d40cb780 con 0x7f83d40cc1e0 2023-10-16T08:18:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f83cc001a00 con 0x7f83d40cc1e0 2023-10-16T08:18:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 >> v1:172.21.15.32:6789/0 conn(0x7f83d40cc1e0 legacy=0x7f83d40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 0x7f83c0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0003f00 0x7f83c00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c0004d10 0x7f83c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cda90 con 0x7f83c0004d10 2023-10-16T08:18:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cb4a0 con 0x7f83c0003f00 2023-10-16T08:18:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000b990 con 0x7f83d40cd240 2023-10-16T08:18:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c0004d10 0x7f83c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d2d9d640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 0x7f83c0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83cbfff640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0003f00 0x7f83c00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 -- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 msgr2=0x7f83c0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40cd240 0x7f83c0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 -- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0003f00 msgr2=0x7f83c00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0003f00 0x7f83c00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.693+0000 7f83d359e640 1 -- 172.21.15.32:0/1555496210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0003530 con 0x7f83c0004d10 2023-10-16T08:18:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d359e640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c0004d10 0x7f83c0005190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f83d40b0300 tx=0x7f83cc01d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10390bcbe1bf485 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0ad630 con 0x7f83c0004d10 2023-10-16T08:18:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc0a8070 con 0x7f83c0004d10 2023-10-16T08:18:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d259c640 1 -- 172.21.15.32:0/1555496210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0ada30 con 0x7f83c0004d10 2023-10-16T08:18:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 -- 172.21.15.32:0/1555496210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c0004d10 msgr2=0x7f83c0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 --2- 172.21.15.32:0/1555496210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c0004d10 0x7f83c0005190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f83d40b0300 tx=0x7f83cc01d2c0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d359e640 1 -- 172.21.15.32:0/1555496210 reap_dead start 2023-10-16T08:18:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 -- 172.21.15.32:0/1555496210 shutdown_connections 2023-10-16T08:18:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 -- 172.21.15.32:0/1555496210 >> 172.21.15.32:0/1555496210 conn(0x7f83d406e590 msgr2=0x7f83d406e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 -- 172.21.15.32:0/1555496210 shutdown_connections 2023-10-16T08:18:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 -- 172.21.15.32:0/1555496210 wait complete. 2023-10-16T08:18:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 Processor -- start 2023-10-16T08:18:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.697+0000 7f83d9246640 1 -- start start 2023-10-16T08:18:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0004d10 0x7f83d414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40b50e0 0x7f83d40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 0x7f83d40c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40b5aa0 con 0x7f83d40c82d0 2023-10-16T08:18:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40c8c90 con 0x7f83c0004d10 2023-10-16T08:18:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d414f2d0 con 0x7f83d40b50e0 2023-10-16T08:18:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 0x7f83d40c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 0x7f83d40c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37890/0 (socket says 172.21.15.32:37890) 2023-10-16T08:18:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 -- 172.21.15.32:0/2902814974 learned_addr learned my addr 172.21.15.32:0/2902814974 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d2d9d640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0004d10 0x7f83d414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83cbfff640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40b50e0 0x7f83d40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 -- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40b50e0 msgr2=0x7f83d40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40b50e0 0x7f83d40b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 -- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0004d10 msgr2=0x7f83d414e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0004d10 0x7f83d414e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 -- 172.21.15.32:0/2902814974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d41f62b0 con 0x7f83d40c82d0 2023-10-16T08:18:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d359e640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 0x7f83d40c8750 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f83cc01cd30 tx=0x7f83c0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0017c0 con 0x7f83d40c82d0 2023-10-16T08:18:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc001920 con 0x7f83d40c82d0 2023-10-16T08:18:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc004d70 con 0x7f83d40c82d0 2023-10-16T08:18:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d40c8eb0 con 0x7f83d40c82d0 2023-10-16T08:18:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d41f57d0 con 0x7f83d40c82d0 2023-10-16T08:18:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.701+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d404e170 con 0x7f83d40c82d0 2023-10-16T08:18:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.705+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f83cc0aa030 con 0x7f83d40c82d0 2023-10-16T08:18:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.705+0000 7f83cb7fe640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83b40664a0 0x7f83b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.705+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f83cc11d780 con 0x7f83d40c82d0 2023-10-16T08:18:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.705+0000 7f83d2d9d640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83b40664a0 0x7f83b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.705+0000 7f83d2d9d640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83b40664a0 0x7f83b4068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009c60 tx=0x7f83c4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.709+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f83cc0ea930 con 0x7f83d40c82d0 2023-10-16T08:18:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.925+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f83d40b6850 con 0x7f83d40c82d0 2023-10-16T08:18:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83cb7fe640 1 -- 172.21.15.32:0/2902814974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v11) v1 ==== 216+0+173 (secure 0 0 0) 0x7f83cc0edf50 con 0x7f83d40c82d0 2023-10-16T08:18:19.935 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:18:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:18:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow rwp" 2023-10-16T08:18:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:18:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw pool=testcephfs1_data,allow rw pool=data_new" 2023-10-16T08:18:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:18:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83b40664a0 msgr2=0x7f83b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83b40664a0 0x7f83b4068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009c60 tx=0x7f83c4009190 comp rx=0 tx=0).stop 2023-10-16T08:18:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 msgr2=0x7f83d40c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 0x7f83d40c8750 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f83cc01cd30 tx=0x7f83c0005800 comp rx=0 tx=0).stop 2023-10-16T08:18:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 shutdown_connections 2023-10-16T08:18:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83d40c82d0 0x7f83d40c8750 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83b40664a0 0x7f83b4068960 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f83d40b50e0 0x7f83d40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 --2- 172.21.15.32:0/2902814974 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83c0004d10 0x7f83d414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:19.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 >> 172.21.15.32:0/2902814974 conn(0x7f83d406e590 msgr2=0x7f83d415bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:19.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 shutdown_connections 2023-10-16T08:18:19.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:19.929+0000 7f83d9246640 1 -- 172.21.15.32:0/2902814974 wait complete. 2023-10-16T08:18:19.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:18:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 Processor -- start 2023-10-16T08:18:20.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 -- start start 2023-10-16T08:18:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc3415ca70 0x7efc340d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc340c65f0 con 0x7efc340cf5d0 2023-10-16T08:18:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc340ce820 con 0x7efc3415ca70 2023-10-16T08:18:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3c372640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc340cfa40 con 0x7efc340d3550 2023-10-16T08:18:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc3415ca70 0x7efc340d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc3415ca70 0x7efc340d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37906/0 (socket says 172.21.15.32:37906) 2023-10-16T08:18:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 -- 172.21.15.32:0/2100472520 learned_addr learned my addr 172.21.15.32:0/2100472520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3990d640 1 --2- 172.21.15.32:0/2100472520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 -- 172.21.15.32:0/2100472520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 msgr2=0x7efc340cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 --2- 172.21.15.32:0/2100472520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 -- 172.21.15.32:0/2100472520 >> v1:172.21.15.186:6789/0 conn(0x7efc340cf5d0 legacy=0x7efc340d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.093+0000 7efc3a10e640 1 -- 172.21.15.32:0/2100472520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340cdf80 con 0x7efc3415ca70 2023-10-16T08:18:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3a10e640 1 --2- 172.21.15.32:0/2100472520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc340d3010 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efc30009870 tx=0x7efc3001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3ac695bc9d8e90 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3910c640 1 -- 172.21.15.32:0/2100472520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300aa650 con 0x7efc3415ca70 2023-10-16T08:18:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3910c640 1 -- 172.21.15.32:0/2100472520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc300aa7b0 con 0x7efc3415ca70 2023-10-16T08:18:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3910c640 1 -- 172.21.15.32:0/2100472520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300aab40 con 0x7efc3415ca70 2023-10-16T08:18:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 -- 172.21.15.32:0/2100472520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 msgr2=0x7efc340d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 --2- 172.21.15.32:0/2100472520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc340d3010 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efc30009870 tx=0x7efc3001c4f0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 -- 172.21.15.32:0/2100472520 shutdown_connections 2023-10-16T08:18:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 --2- 172.21.15.32:0/2100472520 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 --2- 172.21.15.32:0/2100472520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc340d3010 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 -- 172.21.15.32:0/2100472520 >> 172.21.15.32:0/2100472520 conn(0x7efc3409bc50 msgr2=0x7efc340d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:20.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 -- 172.21.15.32:0/2100472520 shutdown_connections 2023-10-16T08:18:20.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.097+0000 7efc3c372640 1 -- 172.21.15.32:0/2100472520 wait complete. 2023-10-16T08:18:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 Processor -- start 2023-10-16T08:18:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 -- start start 2023-10-16T08:18:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 0x7efc34154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:20.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc3415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc341554f0 con 0x7efc3415ca70 2023-10-16T08:18:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3415f300 con 0x7efc340cf5d0 2023-10-16T08:18:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3415e6f0 con 0x7efc340d3550 2023-10-16T08:18:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a10e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 0x7efc34154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc3415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a10e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 0x7efc34154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:51534/0 (socket says 172.21.15.32:51534) 2023-10-16T08:18:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a10e640 1 -- 172.21.15.32:0/4085230179 learned_addr learned my addr 172.21.15.32:0/4085230179 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:20.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3990d640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 -- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 msgr2=0x7efc340cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 -- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 msgr2=0x7efc34154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 0x7efc34154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 -- 172.21.15.32:0/4085230179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30009480 con 0x7efc3415ca70 2023-10-16T08:18:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a10e640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 0x7efc34154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3a90f640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc3415f790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c700 tx=0x7efc2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c003b80 con 0x7efc3415ca70 2023-10-16T08:18:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c003ce0 con 0x7efc3415ca70 2023-10-16T08:18:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00aa30 con 0x7efc3415ca70 2023-10-16T08:18:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3415e9d0 con 0x7efc3415ca70 2023-10-16T08:18:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.101+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3415ef90 con 0x7efc3415ca70 2023-10-16T08:18:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.105+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3415f150 con 0x7efc3415ca70 2023-10-16T08:18:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.105+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc2c00ab90 con 0x7efc3415ca70 2023-10-16T08:18:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.105+0000 7efc2affd640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc180664a0 0x7efc18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.105+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7efc2c083db0 con 0x7efc3415ca70 2023-10-16T08:18:20.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.105+0000 7efc3a10e640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc180664a0 0x7efc18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:20.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.109+0000 7efc3a10e640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc180664a0 0x7efc18068960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc30009870 tx=0x7efc30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:20.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.109+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc2c050ff0 con 0x7efc3415ca70 2023-10-16T08:18:20.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efc3415f4b0 con 0x7efc3415ca70 2023-10-16T08:18:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc2affd640 1 -- 172.21.15.32:0/4085230179 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+97 (secure 0 0 0) 0x7efc3415f4b0 con 0x7efc3415ca70 2023-10-16T08:18:20.308 INFO:teuthology.orchestra.run.smithi032.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data data_new ] 2023-10-16T08:18:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc180664a0 msgr2=0x7efc18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc180664a0 0x7efc18068960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc30009870 tx=0x7efc30009040 comp rx=0 tx=0).stop 2023-10-16T08:18:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 msgr2=0x7efc3415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc3415f790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c700 tx=0x7efc2c00cbc0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 shutdown_connections 2023-10-16T08:18:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc3415ca70 0x7efc3415f790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc180664a0 0x7efc18068960 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc340d3550 0x7efc340cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 --2- 172.21.15.32:0/4085230179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc340cf5d0 0x7efc34154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.301+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 >> 172.21.15.32:0/4085230179 conn(0x7efc3409bc50 msgr2=0x7efc3406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.305+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 shutdown_connections 2023-10-16T08:18:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:20.305+0000 7efc3c372640 1 -- 172.21.15.32:0/4085230179 wait complete. 2023-10-16T08:18:20.329 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:18:20.329 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:18:20.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:18:20.330 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:18:20.330 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:20.330 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:18:20.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:18:20.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:18:20.331 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:20.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.332 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:18:20.343 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:18:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:18:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:18:20.343 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:18:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.344 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:20.344 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:18:20.345 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:20.345 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:18:20.345 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:18:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:18:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:18:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42305sec preferred_lft 42305sec 2023-10-16T08:18:20.346 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:18:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.347 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:20.347 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:18:20.347 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:18:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:18:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.348 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:18:20.348 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:18:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:18:20.349 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.349 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:18:20.350 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:20.350 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:18:20.350 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.350 INFO:teuthology.orchestra.run.smithi032.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:18:20.350 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:20.351 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T08:18:20.351 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:20.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.352 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:18:20.409 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:18:20.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.411 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:18:20.463 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:18:20.463 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:18:20.464 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:18:20.464 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:18:20.464 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:18:20.464 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:18:20.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.465 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:18:20.465 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:18:20.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:18:20.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:18:20.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:18:20.467 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:18:20.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.558 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:18:20.568 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:18:20.568 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:18:20.568 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:20.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:18:20.629 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:18:20.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:18:20.630 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:20.681 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:20.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:18:20.682 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:20.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:18:20.807 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:18:20.863 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:18:20.863 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:18:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:18:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:18:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:18:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:18:20.865 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:18:20.865 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:18:20.865 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:18:20.865 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:18:20.865 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:18:20.866 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:18:20.866 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:18:20.866 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:18:20.866 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:18:20.866 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:18:20.867 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:18:20.867 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:18:20.867 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:18:20.867 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:18:20.867 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:18:20.868 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:18:20.868 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:18:20.868 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:18:20.868 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:18:20.869 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:18:20.869 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:18:20.869 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:18:20.869 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:18:20.869 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:20.870 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:20.870 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:20.870 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:20.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:20.871 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:18:20.921 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:18:20.921 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:18:20.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:18:20.990 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:18:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:18:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:18:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:18:21.044 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:18:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:18:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:18:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:18:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:18:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:18:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:18:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:18:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:18:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:18:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:18:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:18:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:18:21.048 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:18:21.048 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:18:21.048 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:18:21.048 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:18:21.048 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:18:21.049 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:18:21.049 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:18:21.049 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:18:21.049 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:18:21.049 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:18:21.050 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:18:21.050 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:18:21.050 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:18:21.050 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:21.051 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:21.051 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:21.051 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:21.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:21.052 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:18:21.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:18:21.061+0000 7f726ac8d580 -1 init, newargv = 0x563422703310 newargc=15 2023-10-16T08:18:21.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[49267]: starting ceph client 2023-10-16T08:18:21.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[49267]: starting fuse 2023-10-16T08:18:21.077 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:18:21.078 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:18:21.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:18:21.079 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:18:21.079 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:18:21.079 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:18:21.080 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:18:21.080 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:21.080 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:18:21.081 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:18:21.081 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:18:21.081 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:18:21.082 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:21.082 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:18:21.082 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:18:21.083 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:18:21.083 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:18:21.083 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:21.084 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:18:21.084 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:18:21.084 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:18:21.085 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:18:21.085 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:18:21.086 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:18:21.086 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:18:21.086 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:18:21.087 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:21.087 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:18:21.088 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:18:21.208 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.49267.asok 2023-10-16T08:18:21.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:18:21.219 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49267.asok 2023-10-16T08:18:21.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:21.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49267.asok status 2023-10-16T08:18:21.394 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:18:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:18:21.394 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:18:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:18:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:18:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:18:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:18:21.395 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "49267", 2023-10-16T08:18:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:18:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:18:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:18:21.396 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5928, 2023-10-16T08:18:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:18:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:18:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:18:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 5928 2023-10-16T08:18:21.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:18:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:18:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:18:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3448140144 2023-10-16T08:18:21.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:18:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:18:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:18:21.399 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3448140144 2023-10-16T08:18:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.5928 192.168.0.1:0/3448140144", 2023-10-16T08:18:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3448140144", 2023-10-16T08:18:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:18:21.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 19, 2023-10-16T08:18:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 42, 2023-10-16T08:18:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:18:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:18:21.401 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1" 2023-10-16T08:18:21.401 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:18:21.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:21.409 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:18:21.410 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:21.420 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:18:21.420 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:21.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:21.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:21.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:21.481 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-10-16T08:18:21.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:21.537 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-10-16T08:18:21.595 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:21.596 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:21.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 Processor -- start 2023-10-16T08:18:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 -- start start 2023-10-16T08:18:21.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f86840d15d0 0x7f86840cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 0x7f86840c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840ce770 con 0x7f868415cb90 2023-10-16T08:18:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840d0600 con 0x7f86840d15d0 2023-10-16T08:18:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f868b0d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c6450 con 0x7f86840d3010 2023-10-16T08:18:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f8688e72640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f868415cb90 0x7f86840d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33998/0 (socket says 172.21.15.32:33998) 2023-10-16T08:18:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f8688e72640 1 -- 172.21.15.32:0/3733731138 learned_addr learned my addr 172.21.15.32:0/3733731138 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f8683fff640 1 --2- 172.21.15.32:0/3733731138 >> v2:172.21.15.186:3300/0 conn(0x7f86840d15d0 0x7f86840cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897093484 0 0) 0x7f86840ce770 con 0x7f868415cb90 2023-10-16T08:18:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f8689673640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 0x7f86840c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8674003530 con 0x7f868415cb90 2023-10-16T08:18:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928260230 0 0) 0x7f8674003530 con 0x7f868415cb90 2023-10-16T08:18:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 msgr2=0x7f86840c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 0x7f86840c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 >> v2:172.21.15.186:3300/0 conn(0x7f86840d15d0 msgr2=0x7f86840cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 --2- 172.21.15.32:0/3733731138 >> v2:172.21.15.186:3300/0 conn(0x7f86840d15d0 0x7f86840cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840d08a0 con 0x7f868415cb90 2023-10-16T08:18:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f8678001800 con 0x7f868415cb90 2023-10-16T08:18:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 >> v1:172.21.15.32:6789/0 conn(0x7f868415cb90 legacy=0x7f86840d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f8674003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.825+0000 7f86837fe640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 0x7f86740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8674004d10 0x7f8674005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840d0600 con 0x7f8674003980 2023-10-16T08:18:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c6450 con 0x7f8674004d10 2023-10-16T08:18:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867400b990 con 0x7f86840d3010 2023-10-16T08:18:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f8674003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8683fff640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 0x7f86740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8689673640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8674004d10 0x7f8674005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 -- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 msgr2=0x7f86740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86840d3010 0x7f86740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:21.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 -- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8674004d10 msgr2=0x7f8674005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:21.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8674004d10 0x7f8674005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:21.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 -- 172.21.15.32:0/3733731138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8674003530 con 0x7f8674003980 2023-10-16T08:18:21.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f8688e72640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f8674003da0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f867800c240 tx=0x7f867801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913555494411a43d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:21.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780a5910 con 0x7f8674003980 2023-10-16T08:18:21.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86780a5a70 con 0x7f8674003980 2023-10-16T08:18:21.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.829+0000 7f86837fe640 1 -- 172.21.15.32:0/3733731138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780a5e40 con 0x7f8674003980 2023-10-16T08:18:21.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- 172.21.15.32:0/3733731138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 msgr2=0x7f8674003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:21.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 --2- 172.21.15.32:0/3733731138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f8674003da0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f867800c240 tx=0x7f867801cc30 comp rx=0 tx=0).stop 2023-10-16T08:18:21.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8689673640 1 -- 172.21.15.32:0/3733731138 reap_dead start 2023-10-16T08:18:21.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- 172.21.15.32:0/3733731138 shutdown_connections 2023-10-16T08:18:21.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- 172.21.15.32:0/3733731138 >> 172.21.15.32:0/3733731138 conn(0x7f868409bc50 msgr2=0x7f86840d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:21.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- 172.21.15.32:0/3733731138 shutdown_connections 2023-10-16T08:18:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- 172.21.15.32:0/3733731138 wait complete. 2023-10-16T08:18:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 Processor -- start 2023-10-16T08:18:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- start start 2023-10-16T08:18:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f86841506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86841f5320 0x7f868415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8684156af0 0x7f868415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840b6730 con 0x7f8674003980 2023-10-16T08:18:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840b5970 con 0x7f8684156af0 2023-10-16T08:18:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f868b0d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840b5c50 con 0x7f86841f5320 2023-10-16T08:18:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f86841506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f86841506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37934/0 (socket says 172.21.15.32:37934) 2023-10-16T08:18:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 -- 172.21.15.32:0/1657030914 learned_addr learned my addr 172.21.15.32:0/1657030914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8683fff640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86841f5320 0x7f868415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8689673640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8684156af0 0x7f868415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 -- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86841f5320 msgr2=0x7f868415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86841f5320 0x7f868415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 -- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8684156af0 msgr2=0x7f868415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.833+0000 7f8688e72640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8684156af0 0x7f868415bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f8688e72640 1 -- 172.21.15.32:0/1657030914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840b5f30 con 0x7f8674003980 2023-10-16T08:18:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f8689673640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8684156af0 0x7f868415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f8688e72640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f86841506f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f867800c240 tx=0x7f8674005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:21.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780a5e60 con 0x7f8674003980 2023-10-16T08:18:21.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86780057b0 con 0x7f8674003980 2023-10-16T08:18:21.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840b6210 con 0x7f8674003980 2023-10-16T08:18:21.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678005b60 con 0x7f8674003980 2023-10-16T08:18:21.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840cabc0 con 0x7f8674003980 2023-10-16T08:18:21.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8684074010 con 0x7f8674003980 2023-10-16T08:18:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86780b6070 con 0x7f8674003980 2023-10-16T08:18:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f86817fa640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86680664d0 0x7f8668068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f867811da40 con 0x7f8674003980 2023-10-16T08:18:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.837+0000 7f8683fff640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86680664d0 0x7f8668068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.841+0000 7f8683fff640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86680664d0 0x7f8668068990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8684154bb0 tx=0x7f8670009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:21.841+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86780eabc0 con 0x7f8674003980 2023-10-16T08:18:22.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.009+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f868406cbe0 con 0x7f8674003980 2023-10-16T08:18:22.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.009+0000 7f86817fa640 1 -- 172.21.15.32:0/1657030914 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7f86780ee1e0 con 0x7f8674003980 2023-10-16T08:18:22.016 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:22.016 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:22.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.009+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86680664d0 msgr2=0x7f8668068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.009+0000 7f868b0d6640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86680664d0 0x7f8668068990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8684154bb0 tx=0x7f8670009190 comp rx=0 tx=0).stop 2023-10-16T08:18:22.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.009+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 msgr2=0x7f86841506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.009+0000 7f868b0d6640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f86841506f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f867800c240 tx=0x7f8674005800 comp rx=0 tx=0).stop 2023-10-16T08:18:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 shutdown_connections 2023-10-16T08:18:22.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8684156af0 0x7f868415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86680664d0 0x7f8668068990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86841f5320 0x7f868415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 --2- 172.21.15.32:0/1657030914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8674003980 0x7f86841506f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 >> 172.21.15.32:0/1657030914 conn(0x7f868409bc50 msgr2=0x7f868415afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:22.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 shutdown_connections 2023-10-16T08:18:22.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.013+0000 7f868b0d6640 1 -- 172.21.15.32:0/1657030914 wait complete. 2023-10-16T08:18:22.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:22.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 Processor -- start 2023-10-16T08:18:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 -- start start 2023-10-16T08:18:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c15cc60 0x7f178c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f178c161260 0x7f178c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0cc330 con 0x7f178c0cebd0 2023-10-16T08:18:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0cefe0 con 0x7f178c161260 2023-10-16T08:18:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.177+0000 7f1791bd0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0cb720 con 0x7f178c15cc60 2023-10-16T08:18:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f178c161260 0x7f178c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c15cc60 0x7f178c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f178c161260 0x7f178c0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:51570/0 (socket says 172.21.15.32:51570) 2023-10-16T08:18:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 -- 172.21.15.32:0/3151694653 learned_addr learned my addr 172.21.15.32:0/3151694653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178b7fe640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c15cc60 0x7f178c0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:36732/0 (socket says 172.21.15.32:36732) 2023-10-16T08:18:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 -- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c15cc60 msgr2=0x7f178c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 --2- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c15cc60 0x7f178c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 -- 172.21.15.32:0/3151694653 >> v1:172.21.15.186:6789/0 conn(0x7f178c0cebd0 legacy=0x7f178c0b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 -- 172.21.15.32:0/3151694653 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0cba00 con 0x7f178c161260 2023-10-16T08:18:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178affd640 1 --2- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c161260 0x7f178c0ce690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1780009870 tx=0x7f178001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25386c34a1dc31f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178a7fc640 1 -- 172.21.15.32:0/3151694653 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800aa670 con 0x7f178c161260 2023-10-16T08:18:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178a7fc640 1 -- 172.21.15.32:0/3151694653 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17800aa7d0 con 0x7f178c161260 2023-10-16T08:18:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.181+0000 7f178a7fc640 1 -- 172.21.15.32:0/3151694653 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800aab60 con 0x7f178c161260 2023-10-16T08:18:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c161260 msgr2=0x7f178c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 --2- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c161260 0x7f178c0ce690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1780009870 tx=0x7f178001c510 comp rx=0 tx=0).stop 2023-10-16T08:18:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- 172.21.15.32:0/3151694653 shutdown_connections 2023-10-16T08:18:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 --2- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c161260 0x7f178c0ce690 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 --2- 172.21.15.32:0/3151694653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c15cc60 0x7f178c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- 172.21.15.32:0/3151694653 >> 172.21.15.32:0/3151694653 conn(0x7f178c06e570 msgr2=0x7f178c06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- 172.21.15.32:0/3151694653 shutdown_connections 2023-10-16T08:18:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- 172.21.15.32:0/3151694653 wait complete. 2023-10-16T08:18:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 Processor -- start 2023-10-16T08:18:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- start start 2023-10-16T08:18:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c0cebd0 0x7f178c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c15cc60 0x7f178c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 0x7f178c0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c8ca0 con 0x7f178c161260 2023-10-16T08:18:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c155c20 con 0x7f178c15cc60 2023-10-16T08:18:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f1791bd0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c8ed0 con 0x7f178c0cebd0 2023-10-16T08:18:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f178bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 0x7f178c0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f178bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 0x7f178c0b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37938/0 (socket says 172.21.15.32:37938) 2023-10-16T08:18:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f178bfff640 1 -- 172.21.15.32:0/1445609939 learned_addr learned my addr 172.21.15.32:0/1445609939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f178b7fe640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c0cebd0 0x7f178c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.185+0000 7f178affd640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c15cc60 0x7f178c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178bfff640 1 -- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c0cebd0 msgr2=0x7f178c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178bfff640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c0cebd0 0x7f178c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178bfff640 1 -- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c15cc60 msgr2=0x7f178c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178bfff640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c15cc60 0x7f178c0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178bfff640 1 -- 172.21.15.32:0/1445609939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800094d0 con 0x7f178c161260 2023-10-16T08:18:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178bfff640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 0x7f178c0b12c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f178400c770 tx=0x7f178400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784003b80 con 0x7f178c161260 2023-10-16T08:18:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1784003ce0 con 0x7f178c161260 2023-10-16T08:18:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400aa90 con 0x7f178c161260 2023-10-16T08:18:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0c9170 con 0x7f178c161260 2023-10-16T08:18:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c0c9730 con 0x7f178c161260 2023-10-16T08:18:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c0c9920 con 0x7f178c161260 2023-10-16T08:18:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f178400abf0 con 0x7f178c161260 2023-10-16T08:18:22.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1773fff640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17780664a0 0x7f1778068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f1784084070 con 0x7f178c161260 2023-10-16T08:18:22.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.189+0000 7f178b7fe640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17780664a0 0x7f1778068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.193+0000 7f178b7fe640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17780664a0 0x7f1778068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f177c009af0 tx=0x7f177c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.193+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f17840b5090 con 0x7f178c161260 2023-10-16T08:18:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.349+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f178c04e170 con 0x7f178c161260 2023-10-16T08:18:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.349+0000 7f1773fff640 1 -- 172.21.15.32:0/1445609939 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1784051160 con 0x7f178c161260 2023-10-16T08:18:22.358 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17780664a0 msgr2=0x7f1778068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17780664a0 0x7f1778068960 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f177c009af0 tx=0x7f177c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 msgr2=0x7f178c0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 0x7f178c0b12c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f178400c770 tx=0x7f178400cc30 comp rx=0 tx=0).stop 2023-10-16T08:18:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 shutdown_connections 2023-10-16T08:18:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c161260 0x7f178c0b12c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f178c15cc60 0x7f178c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17780664a0 0x7f1778068960 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 --2- 172.21.15.32:0/1445609939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f178c0cebd0 0x7f178c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 >> 172.21.15.32:0/1445609939 conn(0x7f178c06e570 msgr2=0x7f178c15fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 shutdown_connections 2023-10-16T08:18:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.353+0000 7f1791bd0640 1 -- 172.21.15.32:0/1445609939 wait complete. 2023-10-16T08:18:22.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212ccd6640 1 Processor -- start 2023-10-16T08:18:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212ccd6640 1 -- start start 2023-10-16T08:18:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212ccd6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cb040 0x7f21280ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212ccd6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280cc330 con 0x7f212815cc60 2023-10-16T08:18:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212ccd6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280cb4a0 con 0x7f21280cebd0 2023-10-16T08:18:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212ccd6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21280cefe0 con 0x7f21280cb040 2023-10-16T08:18:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f212815cc60 0x7f2128161260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34006/0 (socket says 172.21.15.32:34006) 2023-10-16T08:18:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.517+0000 7f212659c640 1 -- 172.21.15.32:0/1899722645 learned_addr learned my addr 172.21.15.32:0/1899722645 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f211ffff640 1 --2- 172.21.15.32:0/1899722645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cb040 0x7f21280ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166634900 0 0) 0x7f21280cc330 con 0x7f212815cc60 2023-10-16T08:18:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2114003530 con 0x7f212815cc60 2023-10-16T08:18:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328210314 0 0) 0x7f21280cb4a0 con 0x7f21280cebd0 2023-10-16T08:18:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21280cc330 con 0x7f21280cebd0 2023-10-16T08:18:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2220823475 0 0) 0x7f2114003530 con 0x7f212815cc60 2023-10-16T08:18:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f211ffff640 1 -- 172.21.15.32:0/1899722645 >> v1:172.21.15.186:6789/0 conn(0x7f21280cebd0 legacy=0x7f21280cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f211ffff640 1 -- 172.21.15.32:0/1899722645 >> v1:172.21.15.32:6789/0 conn(0x7f212815cc60 legacy=0x7f2128161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f211ffff640 1 -- 172.21.15.32:0/1899722645 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21280cb720 con 0x7f21280cb040 2023-10-16T08:18:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f211ffff640 1 --2- 172.21.15.32:0/1899722645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cb040 0x7f21280ce690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21100097f0 tx=0x7f211001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1d4d2f32fa0bf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118001680 con 0x7f21280cb040 2023-10-16T08:18:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21100a7050 con 0x7f21280cb040 2023-10-16T08:18:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.521+0000 7f2125d9b640 1 -- 172.21.15.32:0/1899722645 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100ab620 con 0x7f21280cb040 2023-10-16T08:18:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- 172.21.15.32:0/1899722645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cb040 msgr2=0x7f21280ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 --2- 172.21.15.32:0/1899722645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cb040 0x7f21280ce690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21100097f0 tx=0x7f211001c3b0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- 172.21.15.32:0/1899722645 shutdown_connections 2023-10-16T08:18:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 --2- 172.21.15.32:0/1899722645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cb040 0x7f21280ce690 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- 172.21.15.32:0/1899722645 >> 172.21.15.32:0/1899722645 conn(0x7f212806e570 msgr2=0x7f212806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- 172.21.15.32:0/1899722645 shutdown_connections 2023-10-16T08:18:22.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- 172.21.15.32:0/1899722645 wait complete. 2023-10-16T08:18:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 Processor -- start 2023-10-16T08:18:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- start start 2023-10-16T08:18:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21280cb040 0x7f2128151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cebd0 0x7f21280c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 0x7f212815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128152200 con 0x7f212815cc60 2023-10-16T08:18:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280c5e40 con 0x7f21280cb040 2023-10-16T08:18:22.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f212ccd6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21280c60f0 con 0x7f21280cebd0 2023-10-16T08:18:22.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.525+0000 7f2126d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 0x7f212815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f212659c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21280cb040 0x7f2128151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 0x7f212815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37952/0 (socket says 172.21.15.32:37952) 2023-10-16T08:18:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 -- 172.21.15.32:0/3699671512 learned_addr learned my addr 172.21.15.32:0/3699671512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f211ffff640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cebd0 0x7f21280c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 -- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cebd0 msgr2=0x7f21280c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cebd0 0x7f21280c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 -- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21280cb040 msgr2=0x7f2128151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21280cb040 0x7f2128151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 -- 172.21.15.32:0/3699671512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009420 con 0x7f212815cc60 2023-10-16T08:18:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f2126d9d640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 0x7f212815a2b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f212000c790 tx=0x7f212000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000cea0 con 0x7f212815cc60 2023-10-16T08:18:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2120003b80 con 0x7f212815cc60 2023-10-16T08:18:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000a850 con 0x7f212815cc60 2023-10-16T08:18:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21280c6be0 con 0x7f212815cc60 2023-10-16T08:18:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21280c6620 con 0x7f212815cc60 2023-10-16T08:18:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212804e170 con 0x7f212815cc60 2023-10-16T08:18:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f212000a9b0 con 0x7f212815cc60 2023-10-16T08:18:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.529+0000 7f211effd640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664d0 0x7f2108068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.533+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f2120083ef0 con 0x7f212815cc60 2023-10-16T08:18:22.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.533+0000 7f212659c640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664d0 0x7f2108068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.533+0000 7f212659c640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664d0 0x7f2108068990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2118000f70 tx=0x7f2118001700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.533+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2120051120 con 0x7f212815cc60 2023-10-16T08:18:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.689+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21280b6830 con 0x7f212815cc60 2023-10-16T08:18:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.689+0000 7f211effd640 1 -- 172.21.15.32:0/3699671512 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7f2120054740 con 0x7f212815cc60 2023-10-16T08:18:22.697 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664d0 msgr2=0x7f2108068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664d0 0x7f2108068990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2118000f70 tx=0x7f2118001700 comp rx=0 tx=0).stop 2023-10-16T08:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 msgr2=0x7f212815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 0x7f212815a2b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f212000c790 tx=0x7f212000cc50 comp rx=0 tx=0).stop 2023-10-16T08:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 shutdown_connections 2023-10-16T08:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f212815cc60 0x7f212815a2b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664d0 0x7f2108068990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f21280cebd0 0x7f21280c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 --2- 172.21.15.32:0/3699671512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21280cb040 0x7f2128151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 >> 172.21.15.32:0/3699671512 conn(0x7f212806e570 msgr2=0x7f212815f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 shutdown_connections 2023-10-16T08:18:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.693+0000 7f212ccd6640 1 -- 172.21.15.32:0/3699671512 wait complete. 2023-10-16T08:18:22.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.857+0000 7f3d2766b640 1 Processor -- start 2023-10-16T08:18:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.857+0000 7f3d2766b640 1 -- start start 2023-10-16T08:18:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d2766b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3d2006a0f0 0x7f3d200c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d2766b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d2006a4d0 0x7f3d200ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d2766b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2006c6d0 con 0x7f3d20164dc0 2023-10-16T08:18:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d2766b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2006c950 con 0x7f3d2006a4d0 2023-10-16T08:18:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d2766b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d201654f0 con 0x7f3d2006a0f0 2023-10-16T08:18:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d2006a4d0 0x7f3d200ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d2006a4d0 0x7f3d200ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33504/0 (socket says 172.21.15.32:33504) 2023-10-16T08:18:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 -- 172.21.15.32:0/3558701392 learned_addr learned my addr 172.21.15.32:0/3558701392 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d25407640 1 --2- 172.21.15.32:0/3558701392 >> v2:172.21.15.186:3300/0 conn(0x7f3d2006a0f0 0x7f3d200c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 -- 172.21.15.32:0/3558701392 >> v2:172.21.15.186:3300/0 conn(0x7f3d2006a0f0 msgr2=0x7f3d200c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 --2- 172.21.15.32:0/3558701392 >> v2:172.21.15.186:3300/0 conn(0x7f3d2006a0f0 0x7f3d200c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 -- 172.21.15.32:0/3558701392 >> v1:172.21.15.186:6789/0 conn(0x7f3d20164dc0 legacy=0x7f3d200ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.861+0000 7f3d24c06640 1 -- 172.21.15.32:0/3558701392 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d201657d0 con 0x7f3d2006a4d0 2023-10-16T08:18:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d24c06640 1 --2- 172.21.15.32:0/3558701392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d2006a4d0 0x7f3d200ccd00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3d200713e0 tx=0x7f3d1001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6239ef660a42127c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d17fff640 1 -- 172.21.15.32:0/3558701392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d100aa620 con 0x7f3d2006a4d0 2023-10-16T08:18:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d17fff640 1 -- 172.21.15.32:0/3558701392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d100aa780 con 0x7f3d2006a4d0 2023-10-16T08:18:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d17fff640 1 -- 172.21.15.32:0/3558701392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d100aab10 con 0x7f3d2006a4d0 2023-10-16T08:18:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 -- 172.21.15.32:0/3558701392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d2006a4d0 msgr2=0x7f3d200ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 --2- 172.21.15.32:0/3558701392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d2006a4d0 0x7f3d200ccd00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3d200713e0 tx=0x7f3d1001c4c0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 -- 172.21.15.32:0/3558701392 shutdown_connections 2023-10-16T08:18:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 --2- 172.21.15.32:0/3558701392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d2006a4d0 0x7f3d200ccd00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 --2- 172.21.15.32:0/3558701392 >> v2:172.21.15.186:3300/0 conn(0x7f3d2006a0f0 0x7f3d200c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 -- 172.21.15.32:0/3558701392 >> 172.21.15.32:0/3558701392 conn(0x7f3d200b0da0 msgr2=0x7f3d200ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:22.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 -- 172.21.15.32:0/3558701392 shutdown_connections 2023-10-16T08:18:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 -- 172.21.15.32:0/3558701392 wait complete. 2023-10-16T08:18:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 Processor -- start 2023-10-16T08:18:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.865+0000 7f3d2766b640 1 -- start start 2023-10-16T08:18:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d2006a0f0 0x7f3d2015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d2006a4d0 0x7f3d200c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 0x7f3d200c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200c4380 con 0x7f3d20164dc0 2023-10-16T08:18:22.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200b26c0 con 0x7f3d2006a0f0 2023-10-16T08:18:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200b2960 con 0x7f3d2006a4d0 2023-10-16T08:18:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 0x7f3d200c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 0x7f3d200c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33506/0 (socket says 172.21.15.32:33506) 2023-10-16T08:18:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 -- 172.21.15.32:0/2930115403 learned_addr learned my addr 172.21.15.32:0/2930115403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25407640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d2006a0f0 0x7f3d2015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d24c06640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d2006a4d0 0x7f3d200c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 -- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d2006a4d0 msgr2=0x7f3d200c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d2006a4d0 0x7f3d200c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 -- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d2006a0f0 msgr2=0x7f3d2015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d2006a0f0 0x7f3d2015ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 -- 172.21.15.32:0/2930115403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d100094d0 con 0x7f3d20164dc0 2023-10-16T08:18:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d25c08640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 0x7f3d200c4810 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b970 tx=0x7f3d1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800cb40 con 0x7f3d20164dc0 2023-10-16T08:18:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1800cca0 con 0x7f3d20164dc0 2023-10-16T08:18:22.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18014680 con 0x7f3d20164dc0 2023-10-16T08:18:22.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d200c4010 con 0x7f3d20164dc0 2023-10-16T08:18:22.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d200b2c40 con 0x7f3d20164dc0 2023-10-16T08:18:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2004e170 con 0x7f3d20164dc0 2023-10-16T08:18:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.869+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3d1801d3b0 con 0x7f3d20164dc0 2023-10-16T08:18:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.873+0000 7f3d15ffb640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3d040664a0 0x7f3d04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.873+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f3d18083f40 con 0x7f3d20164dc0 2023-10-16T08:18:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.873+0000 7f3d25407640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3d040664a0 0x7f3d04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.873+0000 7f3d25407640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3d040664a0 0x7f3d04068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c007a60 tx=0x7f3d1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:22.877+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3d18051170 con 0x7f3d20164dc0 2023-10-16T08:18:23.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.033+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f3d200c41a0 con 0x7f3d20164dc0 2023-10-16T08:18:23.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d15ffb640 1 -- 172.21.15.32:0/2930115403 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7f3d18054790 con 0x7f3d20164dc0 2023-10-16T08:18:23.042 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:23.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3d040664a0 msgr2=0x7f3d04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3d040664a0 0x7f3d04068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c007a60 tx=0x7f3d1c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:23.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 msgr2=0x7f3d200c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 0x7f3d200c4810 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b970 tx=0x7f3d1800be30 comp rx=0 tx=0).stop 2023-10-16T08:18:23.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 shutdown_connections 2023-10-16T08:18:23.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d20164dc0 0x7f3d200c4810 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3d040664a0 0x7f3d04068960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3d2006a4d0 0x7f3d200c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 --2- 172.21.15.32:0/2930115403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d2006a0f0 0x7f3d2015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 >> 172.21.15.32:0/2930115403 conn(0x7f3d200b0da0 msgr2=0x7f3d2006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:23.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 shutdown_connections 2023-10-16T08:18:23.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.037+0000 7f3d2766b640 1 -- 172.21.15.32:0/2930115403 wait complete. 2023-10-16T08:18:23.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbf091640 1 Processor -- start 2023-10-16T08:18:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbf091640 1 -- start start 2023-10-16T08:18:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbf091640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80b84e0 0x7fedb80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbf091640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80d1670 con 0x7fedb80d0210 2023-10-16T08:18:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbf091640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80d1950 con 0x7fedb815cba0 2023-10-16T08:18:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbf091640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80d1460 con 0x7fedb80b84e0 2023-10-16T08:18:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedb7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fedb80d0210 0x7fedb80b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40944/0 (socket says 172.21.15.32:40944) 2023-10-16T08:18:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedb7fff640 1 -- 172.21.15.32:0/3056591512 learned_addr learned my addr 172.21.15.32:0/3056591512 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbd62e640 1 --2- 172.21.15.32:0/3056591512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80b84e0 0x7fedb80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421985389 0 0) 0x7fedb80d1670 con 0x7fedb80d0210 2023-10-16T08:18:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda8003530 con 0x7fedb80d0210 2023-10-16T08:18:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758878151 0 0) 0x7fedb80d1950 con 0x7fedb815cba0 2023-10-16T08:18:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb80d1670 con 0x7fedb815cba0 2023-10-16T08:18:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbd62e640 1 -- 172.21.15.32:0/3056591512 >> v1:172.21.15.186:6789/0 conn(0x7fedb815cba0 legacy=0x7fedb80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbd62e640 1 -- 172.21.15.32:0/3056591512 >> v1:172.21.15.32:6789/0 conn(0x7fedb80d0210 legacy=0x7fedb80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.205+0000 7fedbd62e640 1 -- 172.21.15.32:0/3056591512 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80d1c30 con 0x7fedb80b84e0 2023-10-16T08:18:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbd62e640 1 --2- 172.21.15.32:0/3056591512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80b84e0 0x7fedb80d0f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009dc0 tx=0x7fedb001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f7647a59f086db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00a5a30 con 0x7fedb80b84e0 2023-10-16T08:18:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00a5b90 con 0x7fedb80b84e0 2023-10-16T08:18:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedb77fe640 1 -- 172.21.15.32:0/3056591512 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00ad440 con 0x7fedb80b84e0 2023-10-16T08:18:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbf091640 1 -- 172.21.15.32:0/3056591512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80b84e0 msgr2=0x7fedb80d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbf091640 1 --2- 172.21.15.32:0/3056591512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80b84e0 0x7fedb80d0f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009dc0 tx=0x7fedb001c970 comp rx=0 tx=0).stop 2023-10-16T08:18:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbf091640 1 -- 172.21.15.32:0/3056591512 shutdown_connections 2023-10-16T08:18:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbf091640 1 --2- 172.21.15.32:0/3056591512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80b84e0 0x7fedb80d0f20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbf091640 1 -- 172.21.15.32:0/3056591512 >> 172.21.15.32:0/3056591512 conn(0x7fedb80719c0 msgr2=0x7fedb8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.209+0000 7fedbf091640 1 -- 172.21.15.32:0/3056591512 shutdown_connections 2023-10-16T08:18:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 -- 172.21.15.32:0/3056591512 wait complete. 2023-10-16T08:18:23.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 Processor -- start 2023-10-16T08:18:23.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 -- start start 2023-10-16T08:18:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb80b84e0 0x7fedb814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80d0210 0x7fedb80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 0x7fedb814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80c8370 con 0x7fedb815cba0 2023-10-16T08:18:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80c8650 con 0x7fedb80b84e0 2023-10-16T08:18:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbf091640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80c8930 con 0x7fedb80d0210 2023-10-16T08:18:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 0x7fedb814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 0x7fedb814fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33516/0 (socket says 172.21.15.32:33516) 2023-10-16T08:18:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 -- 172.21.15.32:0/2048971096 learned_addr learned my addr 172.21.15.32:0/2048971096 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:23.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 -- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80d0210 msgr2=0x7fedb80c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbce2d640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb80b84e0 0x7fedb814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedb7fff640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80d0210 0x7fedb80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80d0210 0x7fedb80c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 -- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb80b84e0 msgr2=0x7fedb814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb80b84e0 0x7fedb814ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 -- 172.21.15.32:0/2048971096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0009a70 con 0x7fedb815cba0 2023-10-16T08:18:23.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbce2d640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb80b84e0 0x7fedb814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:23.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedb7fff640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80d0210 0x7fedb80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:23.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.213+0000 7fedbd62e640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 0x7fedb814fe00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fedb001a5a0 tx=0x7fedb0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00adc80 con 0x7fedb815cba0 2023-10-16T08:18:23.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00adde0 con 0x7fedb815cba0 2023-10-16T08:18:23.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00b9690 con 0x7fedb815cba0 2023-10-16T08:18:23.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb80c8c10 con 0x7fedb815cba0 2023-10-16T08:18:23.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb80c9410 con 0x7fedb815cba0 2023-10-16T08:18:23.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb804e170 con 0x7fedb815cba0 2023-10-16T08:18:23.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fedb00b97f0 con 0x7fedb815cba0 2023-10-16T08:18:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedb57fa640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed9c0664a0 0x7fed9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fedb011f730 con 0x7fedb815cba0 2023-10-16T08:18:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedbce2d640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed9c0664a0 0x7fed9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.217+0000 7fedbce2d640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed9c0664a0 0x7fed9c068960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedac0011f0 tx=0x7fedac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.221+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fedb00ec930 con 0x7fedb815cba0 2023-10-16T08:18:23.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.381+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fedb815a600 con 0x7fedb815cba0 2023-10-16T08:18:23.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.381+0000 7fedb57fa640 1 -- 172.21.15.32:0/2048971096 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7fedb0009450 con 0x7fedb815cba0 2023-10-16T08:18:23.389 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed9c0664a0 msgr2=0x7fed9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed9c0664a0 0x7fed9c068960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedac0011f0 tx=0x7fedac009040 comp rx=0 tx=0).stop 2023-10-16T08:18:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 msgr2=0x7fedb814fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 0x7fedb814fe00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fedb001a5a0 tx=0x7fedb0019710 comp rx=0 tx=0).stop 2023-10-16T08:18:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 shutdown_connections 2023-10-16T08:18:23.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb815cba0 0x7fedb814fe00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fed9c0664a0 0x7fed9c068960 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fedb80d0210 0x7fedb80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 --2- 172.21.15.32:0/2048971096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fedb80b84e0 0x7fedb814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 >> 172.21.15.32:0/2048971096 conn(0x7fedb80719c0 msgr2=0x7fedb806b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:23.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 shutdown_connections 2023-10-16T08:18:23.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.385+0000 7fedbf091640 1 -- 172.21.15.32:0/2048971096 wait complete. 2023-10-16T08:18:23.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.545+0000 7fbdbf7d9640 1 Processor -- start 2023-10-16T08:18:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbf7d9640 1 -- start start 2023-10-16T08:18:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbf7d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbdb80c8cb0 0x7fbdb80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbf7d9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80723d0 con 0x7fbdb8159ba0 2023-10-16T08:18:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbf7d9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80717c0 con 0x7fbdb81597c0 2023-10-16T08:18:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbf7d9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb8071aa0 con 0x7fbdb80c8cb0 2023-10-16T08:18:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbdb8159ba0 0x7fbdb80cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40948/0 (socket says 172.21.15.32:40948) 2023-10-16T08:18:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 -- 172.21.15.32:0/3500748882 learned_addr learned my addr 172.21.15.32:0/3500748882 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbdd76640 1 --2- 172.21.15.32:0/3500748882 >> v2:172.21.15.186:3300/0 conn(0x7fbdb80c8cb0 0x7fbdb80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253751202 0 0) 0x7fbdb80723d0 con 0x7fbdb8159ba0 2023-10-16T08:18:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda4003530 con 0x7fbdb8159ba0 2023-10-16T08:18:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325270309 0 0) 0x7fbda4003530 con 0x7fbdb8159ba0 2023-10-16T08:18:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 >> v2:172.21.15.186:3300/0 conn(0x7fbdb80c8cb0 msgr2=0x7fbdb80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 --2- 172.21.15.32:0/3500748882 >> v2:172.21.15.186:3300/0 conn(0x7fbdb80c8cb0 0x7fbdb80cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 >> v1:172.21.15.186:6789/0 conn(0x7fbdb81597c0 legacy=0x7fbdb80ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8071d80 con 0x7fbdb8159ba0 2023-10-16T08:18:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fbda8001680 con 0x7fbdb8159ba0 2023-10-16T08:18:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 >> v1:172.21.15.32:6789/0 conn(0x7fbdb8159ba0 legacy=0x7fbdb80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbda4003980 0x7fbda4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbda4004380 0x7fbda4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4005050 0x7fbda400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8071aa0 con 0x7fbda4004380 2023-10-16T08:18:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400bfb0 con 0x7fbda4005050 2023-10-16T08:18:23.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400c170 con 0x7fbda4003980 2023-10-16T08:18:23.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbda4004380 0x7fbda4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbd575640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbda4003980 0x7fbda4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbdd76640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4005050 0x7fbda400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 -- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbda4003980 msgr2=0x7fbda4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:23.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbda4003980 0x7fbda4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 -- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4005050 msgr2=0x7fbda400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:23.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4005050 0x7fbda400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbcd74640 1 -- 172.21.15.32:0/3500748882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4003530 con 0x7fbda4004380 2023-10-16T08:18:23.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbdd76640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4005050 0x7fbda400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:23.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.549+0000 7fbdbdd76640 1 -- 172.21.15.32:0/3500748882 reap_dead start 2023-10-16T08:18:23.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbcd74640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbda4004380 0x7fbda4004850 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ce540 tx=0x7fbda801f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d40da8ca4bdf64 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8005ba0 con 0x7fbda4004380 2023-10-16T08:18:23.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8005d00 con 0x7fbda4004380 2023-10-16T08:18:23.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdaffff640 1 -- 172.21.15.32:0/3500748882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8008350 con 0x7fbda4004380 2023-10-16T08:18:23.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3500748882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbda4004380 msgr2=0x7fbda4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbda4004380 0x7fbda4004850 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ce540 tx=0x7fbda801f930 comp rx=0 tx=0).stop 2023-10-16T08:18:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3500748882 shutdown_connections 2023-10-16T08:18:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3500748882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbda4004380 0x7fbda4004850 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3500748882 >> 172.21.15.32:0/3500748882 conn(0x7fbdb806c190 msgr2=0x7fbdb806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3500748882 shutdown_connections 2023-10-16T08:18:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.553+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3500748882 wait complete. 2023-10-16T08:18:23.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 Processor -- start 2023-10-16T08:18:23.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 -- start start 2023-10-16T08:18:23.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4004380 0x7fbdb80b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb80acd40 0x7fbdb80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 0x7fbdb80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b1080 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb81edb30 con 0x7fbda4004380 2023-10-16T08:18:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b1f80 con 0x7fbdb80acd40 2023-10-16T08:18:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 0x7fbdb80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 0x7fbdb80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33526/0 (socket says 172.21.15.32:33526) 2023-10-16T08:18:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 -- 172.21.15.32:0/3378920625 learned_addr learned my addr 172.21.15.32:0/3378920625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbcd74640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb80acd40 0x7fbdb80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbd575640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4004380 0x7fbdb80b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 -- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb80acd40 msgr2=0x7fbdb80c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb80acd40 0x7fbdb80c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 -- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4004380 msgr2=0x7fbdb80b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4004380 0x7fbdb80b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 -- 172.21.15.32:0/3378920625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80b2260 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbdd76640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 0x7fbdb80c7db0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80cdff0 tx=0x7fbdb0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0010050 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0011040 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb80b2540 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0012de0 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.557+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb80b2b00 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.561+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb80b2d20 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.561+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbdb0017440 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.561+0000 7fbdadffb640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd9c0664a0 0x7fbd9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.561+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fbdb0086340 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.561+0000 7fbdbd575640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd9c0664a0 0x7fbd9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.561+0000 7fbdbd575640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd9c0664a0 0x7fbd9c068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c10 tx=0x7fbdb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.565+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbdb0053570 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.729+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbdb8074010 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdadffb640 1 -- 172.21.15.32:0/3378920625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbdb0056b90 con 0x7fbdb80ad1c0 2023-10-16T08:18:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd9c0664a0 msgr2=0x7fbd9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd9c0664a0 0x7fbd9c068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c10 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2023-10-16T08:18:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 msgr2=0x7fbdb80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 0x7fbdb80c7db0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80cdff0 tx=0x7fbdb0009f70 comp rx=0 tx=0).stop 2023-10-16T08:18:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 shutdown_connections 2023-10-16T08:18:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb80ad1c0 0x7fbdb80c7db0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd9c0664a0 0x7fbd9c068960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbdb80acd40 0x7fbdb80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 --2- 172.21.15.32:0/3378920625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbda4004380 0x7fbdb80b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 >> 172.21.15.32:0/3378920625 conn(0x7fbdb806c190 msgr2=0x7fbdb806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 shutdown_connections 2023-10-16T08:18:23.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.733+0000 7fbdbf7d9640 1 -- 172.21.15.32:0/3378920625 wait complete. 2023-10-16T08:18:23.759 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:23.760 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:23.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4573f72640 1 Processor -- start 2023-10-16T08:18:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4573f72640 1 -- start start 2023-10-16T08:18:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4573f72640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f456c15cb90 0x7f456c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4573f72640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c0ce770 con 0x7f456c0d3010 2023-10-16T08:18:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4573f72640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c0d0600 con 0x7f456c0d15d0 2023-10-16T08:18:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4573f72640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c0c6450 con 0x7f456c15cb90 2023-10-16T08:18:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4571d0e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f456c15cb90 0x7f456c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4571d0e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f456c15cb90 0x7f456c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33532/0 (socket says 172.21.15.32:33532) 2023-10-16T08:18:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4571d0e640 1 -- 172.21.15.32:0/944958916 learned_addr learned my addr 172.21.15.32:0/944958916 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f457150d640 1 --1- 172.21.15.32:0/944958916 >> v1:172.21.15.32:6789/0 conn(0x7f456c0d3010 0x7f456c0d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40950/0 (socket says 172.21.15.32:40950) 2023-10-16T08:18:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4571d0e640 1 -- 172.21.15.32:0/944958916 >> v1:172.21.15.186:6789/0 conn(0x7f456c0d15d0 legacy=0x7f456c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:23.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4571d0e640 1 -- 172.21.15.32:0/944958916 >> v1:172.21.15.32:6789/0 conn(0x7f456c0d3010 legacy=0x7f456c0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4571d0e640 1 -- 172.21.15.32:0/944958916 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0d08a0 con 0x7f456c15cb90 2023-10-16T08:18:23.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.949+0000 7f4570d0c640 1 -- 172.21.15.32:0/944958916 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24208514 0 0) 0x7f456c0ce770 con 0x7f456c0d3010 2023-10-16T08:18:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4571d0e640 1 --2- 172.21.15.32:0/944958916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c15cb90 0x7f456c0cfb10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4568006fd0 tx=0x7f456801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8d9f136d5abcf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4570d0c640 1 -- 172.21.15.32:0/944958916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680ab620 con 0x7f456c15cb90 2023-10-16T08:18:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4570d0c640 1 -- 172.21.15.32:0/944958916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45680ab780 con 0x7f456c15cb90 2023-10-16T08:18:23.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4570d0c640 1 -- 172.21.15.32:0/944958916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680abb10 con 0x7f456c15cb90 2023-10-16T08:18:23.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 -- 172.21.15.32:0/944958916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c15cb90 msgr2=0x7f456c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 --2- 172.21.15.32:0/944958916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c15cb90 0x7f456c0cfb10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4568006fd0 tx=0x7f456801c420 comp rx=0 tx=0).stop 2023-10-16T08:18:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 -- 172.21.15.32:0/944958916 shutdown_connections 2023-10-16T08:18:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 --2- 172.21.15.32:0/944958916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c15cb90 0x7f456c0cfb10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 -- 172.21.15.32:0/944958916 >> 172.21.15.32:0/944958916 conn(0x7f456c09bc50 msgr2=0x7f456c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:23.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 -- 172.21.15.32:0/944958916 shutdown_connections 2023-10-16T08:18:23.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.953+0000 7f4573f72640 1 -- 172.21.15.32:0/944958916 wait complete. 2023-10-16T08:18:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 Processor -- start 2023-10-16T08:18:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 -- start start 2023-10-16T08:18:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d15d0 0x7f456c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 0x7f456c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15cb90 0x7f456c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c158d70 con 0x7f456c0d3010 2023-10-16T08:18:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c158f60 con 0x7f456c15cb90 2023-10-16T08:18:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c159240 con 0x7f456c0d15d0 2023-10-16T08:18:23.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 0x7f456c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 0x7f456c15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33534/0 (socket says 172.21.15.32:33534) 2023-10-16T08:18:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 -- 172.21.15.32:0/3621895993 learned_addr learned my addr 172.21.15.32:0/3621895993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4571d0e640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d15d0 0x7f456c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457250f640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15cb90 0x7f456c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 -- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d15d0 msgr2=0x7f456c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d15d0 0x7f456c153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 -- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15cb90 msgr2=0x7f456c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15cb90 0x7f456c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 -- 172.21.15.32:0/3621895993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568009480 con 0x7f456c0d3010 2023-10-16T08:18:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f457150d640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 0x7f456c15b5d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c4b0 tx=0x7f455c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c007480 con 0x7f456c0d3010 2023-10-16T08:18:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c0075e0 con 0x7f456c0d3010 2023-10-16T08:18:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c004a30 con 0x7f456c0d3010 2023-10-16T08:18:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c159520 con 0x7f456c0d3010 2023-10-16T08:18:23.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.957+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c159ae0 con 0x7f456c0d3010 2023-10-16T08:18:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.961+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c074010 con 0x7f456c0d3010 2023-10-16T08:18:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.961+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f455c004b90 con 0x7f456c0d3010 2023-10-16T08:18:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.961+0000 7f45627fc640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f45500664d0 0x7f4550068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.961+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f455c083310 con 0x7f456c0d3010 2023-10-16T08:18:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.961+0000 7f4571d0e640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f45500664d0 0x7f4550068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.961+0000 7f4571d0e640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f45500664d0 0x7f4550068990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45680af000 tx=0x7f45680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:23.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:23.965+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f455c0504c0 con 0x7f456c0d3010 2023-10-16T08:18:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.125+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f456c072950 con 0x7f456c0d3010 2023-10-16T08:18:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.125+0000 7f45627fc640 1 -- 172.21.15.32:0/3621895993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7f455c053ae0 con 0x7f456c0d3010 2023-10-16T08:18:24.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:24.133 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f45500664d0 msgr2=0x7f4550068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f45500664d0 0x7f4550068990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45680af000 tx=0x7f45680a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:24.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 msgr2=0x7f456c15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 0x7f456c15b5d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c4b0 tx=0x7f455c00c970 comp rx=0 tx=0).stop 2023-10-16T08:18:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 shutdown_connections 2023-10-16T08:18:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f456c15cb90 0x7f456c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f456c0d3010 0x7f456c15b5d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f45500664d0 0x7f4550068990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 --2- 172.21.15.32:0/3621895993 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f456c0d15d0 0x7f456c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 >> 172.21.15.32:0/3621895993 conn(0x7f456c09bc50 msgr2=0x7f456c0b7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 shutdown_connections 2023-10-16T08:18:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.129+0000 7f4573f72640 1 -- 172.21.15.32:0/3621895993 wait complete. 2023-10-16T08:18:24.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:24.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:24.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.293+0000 7fc6a3a54640 1 Processor -- start 2023-10-16T08:18:24.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.293+0000 7fc6a3a54640 1 -- start start 2023-10-16T08:18:24.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a3a54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc69c159900 0x7fc69c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a3a54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c06a5a0 0x7fc69c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a3a54640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c0b4390 con 0x7fc69c0b3f00 2023-10-16T08:18:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a3a54640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c0cd740 con 0x7fc69c159900 2023-10-16T08:18:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a3a54640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b4600 con 0x7fc69c06a5a0 2023-10-16T08:18:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc69c159900 0x7fc69c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc69c159900 0x7fc69c06e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33544/0 (socket says 172.21.15.32:33544) 2023-10-16T08:18:24.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1598262655 learned_addr learned my addr 172.21.15.32:0/1598262655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:24.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a1ff1640 1 --2- 172.21.15.32:0/1598262655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c06a5a0 0x7fc69c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1598262655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c06a5a0 msgr2=0x7fc69c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 --2- 172.21.15.32:0/1598262655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c06a5a0 0x7fc69c0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1598262655 >> v1:172.21.15.186:6789/0 conn(0x7fc69c0b3f00 legacy=0x7fc69c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.297+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1598262655 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0c8dd0 con 0x7fc69c159900 2023-10-16T08:18:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a17f0640 1 --2- 172.21.15.32:0/1598262655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c159900 0x7fc69c06e260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc698009870 tx=0x7fc69801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2507686d7ec6e2f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc693fff640 1 -- 172.21.15.32:0/1598262655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980aa670 con 0x7fc69c159900 2023-10-16T08:18:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc693fff640 1 -- 172.21.15.32:0/1598262655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6980aa7d0 con 0x7fc69c159900 2023-10-16T08:18:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc693fff640 1 -- 172.21.15.32:0/1598262655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980aab60 con 0x7fc69c159900 2023-10-16T08:18:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1598262655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c159900 msgr2=0x7fc69c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1598262655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c159900 0x7fc69c06e260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc698009870 tx=0x7fc69801c510 comp rx=0 tx=0).stop 2023-10-16T08:18:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1598262655 shutdown_connections 2023-10-16T08:18:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1598262655 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c06a5a0 0x7fc69c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1598262655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c159900 0x7fc69c06e260 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1598262655 >> 172.21.15.32:0/1598262655 conn(0x7fc69c0b0da0 msgr2=0x7fc69c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1598262655 shutdown_connections 2023-10-16T08:18:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1598262655 wait complete. 2023-10-16T08:18:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 Processor -- start 2023-10-16T08:18:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.301+0000 7fc6a3a54640 1 -- start start 2023-10-16T08:18:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 0x7fc69c0c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc69c0b3f00 0x7fc69c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c159900 0x7fc69c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0c35c0 con 0x7fc69c06a5a0 2023-10-16T08:18:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0c3f90 con 0x7fc69c0b3f00 2023-10-16T08:18:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0c29b0 con 0x7fc69c159900 2023-10-16T08:18:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 0x7fc69c0c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 0x7fc69c0c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33552/0 (socket says 172.21.15.32:33552) 2023-10-16T08:18:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1728917017 learned_addr learned my addr 172.21.15.32:0/1728917017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a1ff1640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c159900 0x7fc69c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a0fef640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc69c0b3f00 0x7fc69c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c159900 msgr2=0x7fc69c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c159900 0x7fc69c0c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc69c0b3f00 msgr2=0x7fc69c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc69c0b3f00 0x7fc69c15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 -- 172.21.15.32:0/1728917017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980094d0 con 0x7fc69c06a5a0 2023-10-16T08:18:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a1ff1640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c159900 0x7fc69c0c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a17f0640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 0x7fc69c0c22b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc6980ac040 tx=0x7fc69801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980b3800 con 0x7fc69c06a5a0 2023-10-16T08:18:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6980b3980 con 0x7fc69c06a5a0 2023-10-16T08:18:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980b3d30 con 0x7fc69c06a5a0 2023-10-16T08:18:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c0c2c90 con 0x7fc69c06a5a0 2023-10-16T08:18:24.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c0c3250 con 0x7fc69c06a5a0 2023-10-16T08:18:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.305+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c04e170 con 0x7fc69c06a5a0 2023-10-16T08:18:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.309+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc6980b2380 con 0x7fc69c06a5a0 2023-10-16T08:18:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.309+0000 7fc691ffb640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc6800664a0 0x7fc680068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.309+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fc69811eb70 con 0x7fc69c06a5a0 2023-10-16T08:18:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.309+0000 7fc6a0fef640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc6800664a0 0x7fc680068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.309+0000 7fc6a0fef640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc6800664a0 0x7fc680068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0c2220 tx=0x7fc68c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.313+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc6980ebda0 con 0x7fc69c06a5a0 2023-10-16T08:18:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc69c06b380 con 0x7fc69c06a5a0 2023-10-16T08:18:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc691ffb640 1 -- 172.21.15.32:0/1728917017 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6980c3020 con 0x7fc69c06a5a0 2023-10-16T08:18:24.476 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc6800664a0 msgr2=0x7fc680068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc6800664a0 0x7fc680068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0c2220 tx=0x7fc68c009040 comp rx=0 tx=0).stop 2023-10-16T08:18:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 msgr2=0x7fc69c0c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 0x7fc69c0c22b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc6980ac040 tx=0x7fc69801ad50 comp rx=0 tx=0).stop 2023-10-16T08:18:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 shutdown_connections 2023-10-16T08:18:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc6800664a0 0x7fc680068960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc69c159900 0x7fc69c0c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc69c0b3f00 0x7fc69c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 --2- 172.21.15.32:0/1728917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc69c06a5a0 0x7fc69c0c22b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 >> 172.21.15.32:0/1728917017 conn(0x7fc69c0b0da0 msgr2=0x7fc69c0ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 shutdown_connections 2023-10-16T08:18:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.469+0000 7fc6a3a54640 1 -- 172.21.15.32:0/1728917017 wait complete. 2023-10-16T08:18:24.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:24.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a4672a640 1 Processor -- start 2023-10-16T08:18:24.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a4672a640 1 -- start start 2023-10-16T08:18:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a4672a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a400ceda0 0x7f2a40167be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a4672a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a400cc330 con 0x7f2a400cb040 2023-10-16T08:18:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a4672a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a400cb4d0 con 0x7f2a4015cc60 2023-10-16T08:18:24.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a4672a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a400cb720 con 0x7f2a400ceda0 2023-10-16T08:18:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a44cc7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a400ceda0 0x7f2a40167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a3f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a400cb040 0x7f2a400ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40954/0 (socket says 172.21.15.32:40954) 2023-10-16T08:18:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a44cc7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a400ceda0 0x7f2a40167be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33568/0 (socket says 172.21.15.32:33568) 2023-10-16T08:18:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/3030257795 learned_addr learned my addr 172.21.15.32:0/3030257795 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a44cc7640 1 -- 172.21.15.32:0/3030257795 >> v1:172.21.15.186:6789/0 conn(0x7f2a4015cc60 legacy=0x7f2a40161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a44cc7640 1 -- 172.21.15.32:0/3030257795 >> v1:172.21.15.32:6789/0 conn(0x7f2a400cb040 legacy=0x7f2a400ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a44cc7640 1 -- 172.21.15.32:0/3030257795 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400cba00 con 0x7f2a400ceda0 2023-10-16T08:18:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.637+0000 7f2a3effd640 1 -- 172.21.15.32:0/3030257795 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583945764 0 0) 0x7f2a400cc330 con 0x7f2a400cb040 2023-10-16T08:18:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a44cc7640 1 --2- 172.21.15.32:0/3030257795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40167be0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009d90 tx=0x7f2a3801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93eb532fad99bd80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a3effd640 1 -- 172.21.15.32:0/3030257795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380a59e0 con 0x7f2a400ceda0 2023-10-16T08:18:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a3effd640 1 -- 172.21.15.32:0/3030257795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a380a5b40 con 0x7f2a400ceda0 2023-10-16T08:18:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a3effd640 1 -- 172.21.15.32:0/3030257795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380ad440 con 0x7f2a400ceda0 2023-10-16T08:18:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 -- 172.21.15.32:0/3030257795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 msgr2=0x7f2a40167be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 --2- 172.21.15.32:0/3030257795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40167be0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009d90 tx=0x7f2a3801c920 comp rx=0 tx=0).stop 2023-10-16T08:18:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 -- 172.21.15.32:0/3030257795 shutdown_connections 2023-10-16T08:18:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 --2- 172.21.15.32:0/3030257795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40167be0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 -- 172.21.15.32:0/3030257795 >> 172.21.15.32:0/3030257795 conn(0x7f2a4006e570 msgr2=0x7f2a4006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 -- 172.21.15.32:0/3030257795 shutdown_connections 2023-10-16T08:18:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.641+0000 7f2a4672a640 1 -- 172.21.15.32:0/3030257795 wait complete. 2023-10-16T08:18:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 Processor -- start 2023-10-16T08:18:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 -- start start 2023-10-16T08:18:24.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a400cb040 0x7f2a40154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a4015cc60 0x7f2a40142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40154680 con 0x7f2a400ceda0 2023-10-16T08:18:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40159240 con 0x7f2a4015cc60 2023-10-16T08:18:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400c7040 con 0x7f2a400cb040 2023-10-16T08:18:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33574/0 (socket says 172.21.15.32:33574) 2023-10-16T08:18:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2557451417 learned_addr learned my addr 172.21.15.32:0/2557451417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3ffff640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a400cb040 0x7f2a40154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a44cc7640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a4015cc60 0x7f2a40142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a400cb040 msgr2=0x7f2a40154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a400cb040 0x7f2a40154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a4015cc60 msgr2=0x7f2a40142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a4015cc60 0x7f2a40142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2557451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a38009a70 con 0x7f2a400ceda0 2023-10-16T08:18:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3f7fe640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40142860 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400c4a0 tx=0x7f2a3400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340074e0 con 0x7f2a400ceda0 2023-10-16T08:18:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a34007640 con 0x7f2a400ceda0 2023-10-16T08:18:24.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34004a30 con 0x7f2a400ceda0 2023-10-16T08:18:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a400c72e0 con 0x7f2a400ceda0 2023-10-16T08:18:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.645+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a40157050 con 0x7f2a400ceda0 2023-10-16T08:18:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.649+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4004e170 con 0x7f2a400ceda0 2023-10-16T08:18:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.649+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2a34004b90 con 0x7f2a400ceda0 2023-10-16T08:18:24.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.649+0000 7f2a3cff9640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2a2c0664a0 0x7f2a2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.649+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f2a340833f0 con 0x7f2a400ceda0 2023-10-16T08:18:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.649+0000 7f2a3ffff640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2a2c0664a0 0x7f2a2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.649+0000 7f2a3ffff640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2a2c0664a0 0x7f2a2c068960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ca330 tx=0x7f2a30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.653+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2a34050590 con 0x7f2a400ceda0 2023-10-16T08:18:24.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.805+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a400c9930 con 0x7f2a400ceda0 2023-10-16T08:18:24.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a3cff9640 1 -- 172.21.15.32:0/2557451417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7f2a340180b0 con 0x7f2a400ceda0 2023-10-16T08:18:24.814 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2a2c0664a0 msgr2=0x7f2a2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2a2c0664a0 0x7f2a2c068960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ca330 tx=0x7f2a30009040 comp rx=0 tx=0).stop 2023-10-16T08:18:24.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 msgr2=0x7f2a40142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40142860 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400c4a0 tx=0x7f2a3400c960 comp rx=0 tx=0).stop 2023-10-16T08:18:24.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 shutdown_connections 2023-10-16T08:18:24.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a4015cc60 0x7f2a40142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a400ceda0 0x7f2a40142860 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2a2c0664a0 0x7f2a2c068960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 --2- 172.21.15.32:0/2557451417 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2a400cb040 0x7f2a40154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 >> 172.21.15.32:0/2557451417 conn(0x7f2a4006e570 msgr2=0x7f2a400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 shutdown_connections 2023-10-16T08:18:24.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.809+0000 7f2a4672a640 1 -- 172.21.15.32:0/2557451417 wait complete. 2023-10-16T08:18:24.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa496e17640 1 Processor -- start 2023-10-16T08:18:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa496e17640 1 -- start start 2023-10-16T08:18:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa496e17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa4900d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa496e17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4900d1670 con 0x7fa49015cba0 2023-10-16T08:18:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa496e17640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4900d1950 con 0x7fa4900d0210 2023-10-16T08:18:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa496e17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900d1460 con 0x7fa4900b84e0 2023-10-16T08:18:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa494bb3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa49015cba0 0x7fa4900cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40968/0 (socket says 172.21.15.32:40968) 2023-10-16T08:18:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa494bb3640 1 -- 172.21.15.32:0/2763317551 learned_addr learned my addr 172.21.15.32:0/2763317551 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa4953b4640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa4900d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509447093 0 0) 0x7fa4900d1670 con 0x7fa49015cba0 2023-10-16T08:18:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa480003530 con 0x7fa49015cba0 2023-10-16T08:18:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3458745662 0 0) 0x7fa480003530 con 0x7fa49015cba0 2023-10-16T08:18:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 msgr2=0x7fa4900d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa4900d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 >> v1:172.21.15.186:6789/0 conn(0x7fa4900d0210 legacy=0x7fa4900b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900d1c30 con 0x7fa49015cba0 2023-10-16T08:18:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fa484001680 con 0x7fa49015cba0 2023-10-16T08:18:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 >> v1:172.21.15.32:6789/0 conn(0x7fa49015cba0 legacy=0x7fa4900cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4800039b0 0x7fa480003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa480004360 0x7fa480004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa480005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900d1460 con 0x7fa480004360 2023-10-16T08:18:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48000b990 con 0x7fa4800039b0 2023-10-16T08:18:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48000bb50 con 0x7fa4900b84e0 2023-10-16T08:18:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48ffff640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa480004360 0x7fa480004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa494bb3640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4800039b0 0x7fa480003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa4953b4640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa480005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48ffff640 1 -- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 msgr2=0x7fa480005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48ffff640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa480005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48ffff640 1 -- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4800039b0 msgr2=0x7fa480003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48ffff640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4800039b0 0x7fa480003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa48ffff640 1 -- 172.21.15.32:0/2763317551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa480003530 con 0x7fa480004360 2023-10-16T08:18:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.977+0000 7fa4953b4640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900b84e0 0x7fa480005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa48ffff640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa480004360 0x7fa480004830 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa4900b5aa0 tx=0x7fa47c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7600d68c8a09ab66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0a8830 con 0x7fa480004360 2023-10-16T08:18:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c0a8990 con 0x7fa480004360 2023-10-16T08:18:24.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa48f7fe640 1 -- 172.21.15.32:0/2763317551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0a8d40 con 0x7fa480004360 2023-10-16T08:18:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa496e17640 1 -- 172.21.15.32:0/2763317551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa480004360 msgr2=0x7fa480004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa496e17640 1 --2- 172.21.15.32:0/2763317551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa480004360 0x7fa480004830 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa4900b5aa0 tx=0x7fa47c01ec10 comp rx=0 tx=0).stop 2023-10-16T08:18:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa4953b4640 1 -- 172.21.15.32:0/2763317551 reap_dead start 2023-10-16T08:18:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa496e17640 1 -- 172.21.15.32:0/2763317551 shutdown_connections 2023-10-16T08:18:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa496e17640 1 -- 172.21.15.32:0/2763317551 >> 172.21.15.32:0/2763317551 conn(0x7fa4900719c0 msgr2=0x7fa490071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa496e17640 1 -- 172.21.15.32:0/2763317551 shutdown_connections 2023-10-16T08:18:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.981+0000 7fa496e17640 1 -- 172.21.15.32:0/2763317551 wait complete. 2023-10-16T08:18:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 Processor -- start 2023-10-16T08:18:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 -- start start 2023-10-16T08:18:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa480004360 0x7fa4900c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900c8de0 0x7fa4901e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 0x7fa490143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49015d9b0 con 0x7fa4901e5a00 2023-10-16T08:18:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901e5f10 con 0x7fa480004360 2023-10-16T08:18:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901e61f0 con 0x7fa4900c8de0 2023-10-16T08:18:24.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 0x7fa490143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 0x7fa490143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33592/0 (socket says 172.21.15.32:33592) 2023-10-16T08:18:24.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 -- 172.21.15.32:0/2493889740 learned_addr learned my addr 172.21.15.32:0/2493889740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:24.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa48ffff640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900c8de0 0x7fa4901e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa494bb3640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa480004360 0x7fa4900c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 -- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900c8de0 msgr2=0x7fa4901e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900c8de0 0x7fa4901e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 -- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa480004360 msgr2=0x7fa4900c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa480004360 0x7fa4900c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 -- 172.21.15.32:0/2493889740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4901e64d0 con 0x7fa4901e5a00 2023-10-16T08:18:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa4953b4640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 0x7fa490143f10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa4900d3b80 tx=0x7fa488009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800e050 con 0x7fa4901e5a00 2023-10-16T08:18:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48800f040 con 0x7fa4901e5a00 2023-10-16T08:18:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4880158a0 con 0x7fa4901e5a00 2023-10-16T08:18:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4901e67b0 con 0x7fa4901e5a00 2023-10-16T08:18:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.985+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4901e6fb0 con 0x7fa4901e5a00 2023-10-16T08:18:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.989+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49004e170 con 0x7fa4901e5a00 2023-10-16T08:18:25.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.989+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa488015a00 con 0x7fa4901e5a00 2023-10-16T08:18:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.989+0000 7fa48d7fa640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa4740664a0 0x7fa474068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.989+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fa488084cd0 con 0x7fa4901e5a00 2023-10-16T08:18:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.989+0000 7fa494bb3640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa4740664a0 0x7fa474068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.989+0000 7fa494bb3640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa4740664a0 0x7fa474068960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa484000c10 tx=0x7fa484009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:24.993+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa488051dd0 con 0x7fa4901e5a00 2023-10-16T08:18:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.149+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7fa4901e6c60 con 0x7fa4901e5a00 2023-10-16T08:18:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.149+0000 7fa48d7fa640 1 -- 172.21.15.32:0/2493889740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7fa4880553f0 con 0x7fa4901e5a00 2023-10-16T08:18:25.156 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa4740664a0 msgr2=0x7fa474068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa4740664a0 0x7fa474068960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa484000c10 tx=0x7fa484009040 comp rx=0 tx=0).stop 2023-10-16T08:18:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 msgr2=0x7fa490143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 0x7fa490143f10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa4900d3b80 tx=0x7fa488009f90 comp rx=0 tx=0).stop 2023-10-16T08:18:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 shutdown_connections 2023-10-16T08:18:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4901e5a00 0x7fa490143f10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa4740664a0 0x7fa474068960 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa4900c8de0 0x7fa4901e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 --2- 172.21.15.32:0/2493889740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa480004360 0x7fa4900c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 >> 172.21.15.32:0/2493889740 conn(0x7fa4900719c0 msgr2=0x7fa4900b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 shutdown_connections 2023-10-16T08:18:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.153+0000 7fa496e17640 1 -- 172.21.15.32:0/2493889740 wait complete. 2023-10-16T08:18:25.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 Processor -- start 2023-10-16T08:18:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 -- start start 2023-10-16T08:18:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f49c415cc60 0x7f49c40cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49c40ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40cc330 con 0x7f49c40cebd0 2023-10-16T08:18:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40cefe0 con 0x7f49c415cc60 2023-10-16T08:18:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c9c54640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40cb720 con 0x7f49c4161260 2023-10-16T08:18:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49c40cebd0 0x7f49c40b2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40976/0 (socket says 172.21.15.32:40976) 2023-10-16T08:18:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.317+0000 7f49c3fff640 1 -- 172.21.15.32:0/1231291675 learned_addr learned my addr 172.21.15.32:0/1231291675 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c37fe640 1 --2- 172.21.15.32:0/1231291675 >> v2:172.21.15.186:3300/0 conn(0x7f49c415cc60 0x7f49c40cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c2ffd640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49c40ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419926868 0 0) 0x7f49c40cc330 con 0x7f49c40cebd0 2023-10-16T08:18:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac003530 con 0x7f49c40cebd0 2023-10-16T08:18:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2140190498 0 0) 0x7f49ac003530 con 0x7f49c40cebd0 2023-10-16T08:18:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 msgr2=0x7f49c40ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49c40ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 >> v2:172.21.15.186:3300/0 conn(0x7f49c415cc60 msgr2=0x7f49c40cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 --2- 172.21.15.32:0/1231291675 >> v2:172.21.15.186:3300/0 conn(0x7f49c415cc60 0x7f49c40cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40cba00 con 0x7f49c40cebd0 2023-10-16T08:18:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f49bc001a00 con 0x7f49c40cebd0 2023-10-16T08:18:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 >> v1:172.21.15.32:6789/0 conn(0x7f49c40cebd0 legacy=0x7f49c40b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49ac003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac003f00 0x7f49ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac004d10 0x7f49ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40cefe0 con 0x7f49ac004d10 2023-10-16T08:18:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40cb720 con 0x7f49ac003f00 2023-10-16T08:18:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00b990 con 0x7f49c4161260 2023-10-16T08:18:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c37fe640 1 --2- 172.21.15.32:0/1231291675 >> v2:172.21.15.186:3300/0 conn(0x7f49c415cc60 0x7f49c40cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac004d10 0x7f49ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c2ffd640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac003f00 0x7f49ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c37fe640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49ac003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c3fff640 1 -- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 msgr2=0x7f49ac003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49ac003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c3fff640 1 -- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac003f00 msgr2=0x7f49ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac003f00 0x7f49ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c3fff640 1 -- 172.21.15.32:0/1231291675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac003530 con 0x7f49ac004d10 2023-10-16T08:18:25.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.321+0000 7f49c37fe640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c4161260 0x7f49ac003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:25.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac004d10 0x7f49ac005190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00a830 tx=0x7f49bc01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd60e8a6f514186e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0ad630 con 0x7f49ac004d10 2023-10-16T08:18:25.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc0a8070 con 0x7f49ac004d10 2023-10-16T08:18:25.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c27fc640 1 -- 172.21.15.32:0/1231291675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0ada30 con 0x7f49ac004d10 2023-10-16T08:18:25.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 -- 172.21.15.32:0/1231291675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac004d10 msgr2=0x7f49ac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 --2- 172.21.15.32:0/1231291675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49ac004d10 0x7f49ac005190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00a830 tx=0x7f49bc01d300 comp rx=0 tx=0).stop 2023-10-16T08:18:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c3fff640 1 -- 172.21.15.32:0/1231291675 reap_dead start 2023-10-16T08:18:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 -- 172.21.15.32:0/1231291675 shutdown_connections 2023-10-16T08:18:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 -- 172.21.15.32:0/1231291675 >> 172.21.15.32:0/1231291675 conn(0x7f49c406e570 msgr2=0x7f49c406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:25.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 -- 172.21.15.32:0/1231291675 shutdown_connections 2023-10-16T08:18:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 -- 172.21.15.32:0/1231291675 wait complete. 2023-10-16T08:18:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 Processor -- start 2023-10-16T08:18:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.325+0000 7f49c9c54640 1 -- start start 2023-10-16T08:18:25.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac004d10 0x7f49c40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c41549c0 0x7f49c40c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 0x7f49c40c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41570f0 con 0x7f49c4154e40 2023-10-16T08:18:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40c9410 con 0x7f49ac004d10 2023-10-16T08:18:25.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41567c0 con 0x7f49c41549c0 2023-10-16T08:18:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 0x7f49c40c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 0x7f49c40c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33614/0 (socket says 172.21.15.32:33614) 2023-10-16T08:18:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 -- 172.21.15.32:0/2252349182 learned_addr learned my addr 172.21.15.32:0/2252349182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c37fe640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac004d10 0x7f49c40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c2ffd640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c41549c0 0x7f49c40c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 -- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c41549c0 msgr2=0x7f49c40c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c41549c0 0x7f49c40c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 -- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac004d10 msgr2=0x7f49c40accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac004d10 0x7f49c40accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 -- 172.21.15.32:0/2252349182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4156aa0 con 0x7f49c4154e40 2023-10-16T08:18:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c2ffd640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c41549c0 0x7f49c40c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c3fff640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 0x7f49c40c9fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f49bc01dba0 tx=0x7f49ac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0017c0 con 0x7f49c4154e40 2023-10-16T08:18:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc001920 con 0x7f49c4154e40 2023-10-16T08:18:25.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc004d70 con 0x7f49c4154e40 2023-10-16T08:18:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c4156d80 con 0x7f49c4154e40 2023-10-16T08:18:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c40c8e60 con 0x7f49c4154e40 2023-10-16T08:18:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c404e170 con 0x7f49c4154e40 2023-10-16T08:18:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.329+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49bc0aa030 con 0x7f49c4154e40 2023-10-16T08:18:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.333+0000 7f49abfff640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f49b00664a0 0x7f49b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.333+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f49bc11d8b0 con 0x7f49c4154e40 2023-10-16T08:18:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.333+0000 7f49c37fe640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f49b00664a0 0x7f49b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.333+0000 7f49c37fe640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f49b00664a0 0x7f49b0068960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49b40093f0 tx=0x7f49b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.337+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49bc0eaaf0 con 0x7f49c4154e40 2023-10-16T08:18:25.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f49c404e170 con 0x7f49c4154e40 2023-10-16T08:18:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49abfff640 1 -- 172.21.15.32:0/2252349182 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f49bc0ee110 con 0x7f49c4154e40 2023-10-16T08:18:25.500 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f49b00664a0 msgr2=0x7f49b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f49b00664a0 0x7f49b0068960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49b40093f0 tx=0x7f49b4008040 comp rx=0 tx=0).stop 2023-10-16T08:18:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 msgr2=0x7f49c40c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 0x7f49c40c9fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f49bc01dba0 tx=0x7f49ac005800 comp rx=0 tx=0).stop 2023-10-16T08:18:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 shutdown_connections 2023-10-16T08:18:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49c4154e40 0x7f49c40c9fc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f49b00664a0 0x7f49b0068960 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f49c41549c0 0x7f49c40c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 --2- 172.21.15.32:0/2252349182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f49ac004d10 0x7f49c40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.493+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 >> 172.21.15.32:0/2252349182 conn(0x7f49c406e570 msgr2=0x7f49c406eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.497+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 shutdown_connections 2023-10-16T08:18:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.497+0000 7f49c9c54640 1 -- 172.21.15.32:0/2252349182 wait complete. 2023-10-16T08:18:25.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.657+0000 7f9011405640 1 Processor -- start 2023-10-16T08:18:25.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.657+0000 7f9011405640 1 -- start start 2023-10-16T08:18:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.657+0000 7f9011405640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 0x7f900c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.657+0000 7f9011405640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c15be60 con 0x7f900c0b07f0 2023-10-16T08:18:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.657+0000 7f9011405640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c15c1f0 con 0x7f900c0b4020 2023-10-16T08:18:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.657+0000 7f9011405640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c15b3b0 con 0x7f900c070c40 2023-10-16T08:18:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f900c0b07f0 0x7f900c164490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40984/0 (socket says 172.21.15.32:40984) 2023-10-16T08:18:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900b7fe640 1 -- 172.21.15.32:0/3728902896 learned_addr learned my addr 172.21.15.32:0/3728902896 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900affd640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 0x7f900c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689876721 0 0) 0x7f900c15be60 con 0x7f900c0b07f0 2023-10-16T08:18:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff8003530 con 0x7f900c0b07f0 2023-10-16T08:18:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241830546 0 0) 0x7f900c15c1f0 con 0x7f900c0b4020 2023-10-16T08:18:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c15be60 con 0x7f900c0b4020 2023-10-16T08:18:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162292992 0 0) 0x7f8ff8003530 con 0x7f900c0b07f0 2023-10-16T08:18:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 msgr2=0x7f900c15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 0x7f900c15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 >> v1:172.21.15.186:6789/0 conn(0x7f900c0b4020 legacy=0x7f900c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c071020 con 0x7f900c0b07f0 2023-10-16T08:18:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f9004001a00 con 0x7f900c0b07f0 2023-10-16T08:18:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 >> v1:172.21.15.32:6789/0 conn(0x7f900c0b07f0 legacy=0x7f900c164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff8003b40 0x7f8ff8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff80047d0 0x7f8ff8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 0x7f8ff80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c15b3b0 con 0x7f8ff80047d0 2023-10-16T08:18:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff800baa0 con 0x7f8ff8003b40 2023-10-16T08:18:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff800bc60 con 0x7f900c070c40 2023-10-16T08:18:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f9003fff640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff80047d0 0x7f8ff8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900b7fe640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 0x7f8ff80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f900affd640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff8003b40 0x7f8ff8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f9003fff640 1 -- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 msgr2=0x7f8ff80051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f9003fff640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f900c070c40 0x7f8ff80051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f9003fff640 1 -- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff8003b40 msgr2=0x7f8ff8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f9003fff640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ff8003b40 0x7f8ff8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.661+0000 7f9003fff640 1 -- 172.21.15.32:0/3728902896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8003530 con 0x7f8ff80047d0 2023-10-16T08:18:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9003fff640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff80047d0 0x7f8ff8004c50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40018d0 tx=0x7f8ff401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3767b2eb9488f6b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40052d0 con 0x7f8ff80047d0 2023-10-16T08:18:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff4005450 con 0x7f8ff80047d0 2023-10-16T08:18:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f900a7fc640 1 -- 172.21.15.32:0/3728902896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4003840 con 0x7f8ff80047d0 2023-10-16T08:18:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 -- 172.21.15.32:0/3728902896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff80047d0 msgr2=0x7f8ff8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 --2- 172.21.15.32:0/3728902896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff80047d0 0x7f8ff8004c50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40018d0 tx=0x7f8ff401c560 comp rx=0 tx=0).stop 2023-10-16T08:18:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f900b7fe640 1 -- 172.21.15.32:0/3728902896 reap_dead start 2023-10-16T08:18:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 -- 172.21.15.32:0/3728902896 shutdown_connections 2023-10-16T08:18:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 -- 172.21.15.32:0/3728902896 >> 172.21.15.32:0/3728902896 conn(0x7f900c06c190 msgr2=0x7f900c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 -- 172.21.15.32:0/3728902896 shutdown_connections 2023-10-16T08:18:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 -- 172.21.15.32:0/3728902896 wait complete. 2023-10-16T08:18:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 Processor -- start 2023-10-16T08:18:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.665+0000 7f9011405640 1 -- start start 2023-10-16T08:18:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ff80047d0 0x7f900c14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2cf0 0x7f900c14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 0x7f900c0c2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b2860 con 0x7f900c0b3170 2023-10-16T08:18:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c14c290 con 0x7f900c0b2cf0 2023-10-16T08:18:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b1c50 con 0x7f8ff80047d0 2023-10-16T08:18:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 0x7f900c0c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 0x7f900c0c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33640/0 (socket says 172.21.15.32:33640) 2023-10-16T08:18:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 -- 172.21.15.32:0/3982323608 learned_addr learned my addr 172.21.15.32:0/3982323608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9003fff640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2cf0 0x7f900c14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900affd640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ff80047d0 0x7f900c14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 -- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ff80047d0 msgr2=0x7f900c14bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ff80047d0 0x7f900c14bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 -- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2cf0 msgr2=0x7f900c14c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2cf0 0x7f900c14c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 -- 172.21.15.32:0/3982323608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0b1f30 con 0x7f900c0b3170 2023-10-16T08:18:25.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f900b7fe640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 0x7f900c0c2190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9004000f90 tx=0x7f9004007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004001370 con 0x7f900c0b3170 2023-10-16T08:18:25.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400aa90 con 0x7f900c0b3170 2023-10-16T08:18:25.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c0b2210 con 0x7f900c0b3170 2023-10-16T08:18:25.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400ae40 con 0x7f900c0b3170 2023-10-16T08:18:25.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c0af750 con 0x7f900c0b3170 2023-10-16T08:18:25.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c074010 con 0x7f900c0b3170 2023-10-16T08:18:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.669+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f90040133f0 con 0x7f900c0b3170 2023-10-16T08:18:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.673+0000 7f90037fe640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fec0664a0 0x7f8fec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.673+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f90040849a0 con 0x7f900c0b3170 2023-10-16T08:18:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.673+0000 7f900affd640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fec0664a0 0x7f8fec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.673+0000 7f900affd640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fec0664a0 0x7f8fec068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f900c06d200 tx=0x7f8ffc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:25.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.673+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9004051a90 con 0x7f900c0b3170 2023-10-16T08:18:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.849+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f900c0b29a0 con 0x7f900c0b3170 2023-10-16T08:18:25.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.849+0000 7f90037fe640 1 -- 172.21.15.32:0/3982323608 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9004017020 con 0x7f900c0b3170 2023-10-16T08:18:25.857 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:25.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fec0664a0 msgr2=0x7f8fec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fec0664a0 0x7f8fec068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f900c06d200 tx=0x7f8ffc009190 comp rx=0 tx=0).stop 2023-10-16T08:18:25.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 msgr2=0x7f900c0c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:25.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 0x7f900c0c2190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9004000f90 tx=0x7f9004007a60 comp rx=0 tx=0).stop 2023-10-16T08:18:25.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 shutdown_connections 2023-10-16T08:18:25.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900c0b3170 0x7f900c0c2190 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f900c0b2cf0 0x7f900c14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8fec0664a0 0x7f8fec068960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 --2- 172.21.15.32:0/3982323608 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8ff80047d0 0x7f900c14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 >> 172.21.15.32:0/3982323608 conn(0x7f900c06c190 msgr2=0x7f900c15daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 shutdown_connections 2023-10-16T08:18:25.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:25.853+0000 7f9011405640 1 -- 172.21.15.32:0/3982323608 wait complete. 2023-10-16T08:18:25.878 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:18:25.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:25.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:25.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[49267]: fuse finished with error 0 and tester_r 0 2023-10-16T08:18:25.901 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:32.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:18:32.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:32.003 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:18:32.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=testcephfs1_data' 2023-10-16T08:18:32.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.213+0000 7f796f53e640 1 Processor -- start 2023-10-16T08:18:32.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.213+0000 7f796f53e640 1 -- start start 2023-10-16T08:18:32.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796f53e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f79680b85d0 0x7f796815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:32.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796f53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f79680cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:32.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796f53e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796815e1d0 con 0x7f79680d0130 2023-10-16T08:18:32.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796f53e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680d05a0 con 0x7f79680b85d0 2023-10-16T08:18:32.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796f53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7968161720 con 0x7f79680b89b0 2023-10-16T08:18:32.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796d2da640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f79680b85d0 0x7f796815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:32.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796cad9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f79680cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:32.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796dadb640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79680d0130 0x7f79680d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40998/0 (socket says 172.21.15.32:40998) 2023-10-16T08:18:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796dadb640 1 -- 172.21.15.32:0/1272439260 learned_addr learned my addr 172.21.15.32:0/1272439260 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f795ffff640 1 -- 172.21.15.32:0/1272439260 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567446637 0 0) 0x7f796815e1d0 con 0x7f79680d0130 2023-10-16T08:18:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f795ffff640 1 -- 172.21.15.32:0/1272439260 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7954003530 con 0x7f79680d0130 2023-10-16T08:18:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796d2da640 1 -- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 msgr2=0x7f79680cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796d2da640 1 --2- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f79680cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796d2da640 1 -- 172.21.15.32:0/1272439260 >> v1:172.21.15.32:6789/0 conn(0x7f79680d0130 legacy=0x7f79680d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:32.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796d2da640 1 -- 172.21.15.32:0/1272439260 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968160ba0 con 0x7f79680b85d0 2023-10-16T08:18:32.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.217+0000 7f796cad9640 1 --2- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f79680cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:32.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796d2da640 1 --2- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680b85d0 0x7f796815dc90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f79640097c0 tx=0x7f796401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10573eb35e829d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:32.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f795ffff640 1 -- 172.21.15.32:0/1272439260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640ab650 con 0x7f79680b85d0 2023-10-16T08:18:32.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f795ffff640 1 -- 172.21.15.32:0/1272439260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79640ab7b0 con 0x7f79680b85d0 2023-10-16T08:18:32.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f795ffff640 1 -- 172.21.15.32:0/1272439260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640abb40 con 0x7f79680b85d0 2023-10-16T08:18:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 -- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680b85d0 msgr2=0x7f796815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 --2- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680b85d0 0x7f796815dc90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f79640097c0 tx=0x7f796401c450 comp rx=0 tx=0).stop 2023-10-16T08:18:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 -- 172.21.15.32:0/1272439260 shutdown_connections 2023-10-16T08:18:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 --2- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f79680cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 --2- 172.21.15.32:0/1272439260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680b85d0 0x7f796815dc90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 -- 172.21.15.32:0/1272439260 >> 172.21.15.32:0/1272439260 conn(0x7f796806b750 msgr2=0x7f796806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:32.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 -- 172.21.15.32:0/1272439260 shutdown_connections 2023-10-16T08:18:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.221+0000 7f796f53e640 1 -- 172.21.15.32:0/1272439260 wait complete. 2023-10-16T08:18:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 Processor -- start 2023-10-16T08:18:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 -- start start 2023-10-16T08:18:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 0x7f79680c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f796814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680d0130 0x7f79680b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680cbaa0 con 0x7f79680b85d0 2023-10-16T08:18:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680ca270 con 0x7f79680d0130 2023-10-16T08:18:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79681504d0 con 0x7f79680b89b0 2023-10-16T08:18:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 0x7f79680c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 0x7f79680c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33654/0 (socket says 172.21.15.32:33654) 2023-10-16T08:18:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 -- 172.21.15.32:0/3777624934 learned_addr learned my addr 172.21.15.32:0/3777624934 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796cad9640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f796814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796dadb640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680d0130 0x7f79680b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 -- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 msgr2=0x7f796814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f796814fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 -- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680d0130 msgr2=0x7f79680b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680d0130 0x7f79680b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 -- 172.21.15.32:0/3777624934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958009420 con 0x7f79680b85d0 2023-10-16T08:18:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796d2da640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 0x7f79680c9d30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f79640ad040 tx=0x7f796401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640bf610 con 0x7f79680b85d0 2023-10-16T08:18:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79640bf770 con 0x7f79680b85d0 2023-10-16T08:18:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640bfb20 con 0x7f79680b85d0 2023-10-16T08:18:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7964009420 con 0x7f79680b85d0 2023-10-16T08:18:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.225+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7968150a90 con 0x7f79680b85d0 2023-10-16T08:18:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.229+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968150c80 con 0x7f79680b85d0 2023-10-16T08:18:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.229+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f79640bfc80 con 0x7f79680b85d0 2023-10-16T08:18:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.229+0000 7f795dffb640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f794c0664a0 0x7f794c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.229+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f796411eaa0 con 0x7f79680b85d0 2023-10-16T08:18:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.229+0000 7f796cad9640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f794c0664a0 0x7f794c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.229+0000 7f796cad9640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f794c0664a0 0x7f794c068960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f796815e490 tx=0x7f7958009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:32.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.233+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f79640ebcf0 con 0x7f79680b85d0 2023-10-16T08:18:32.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.449+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]} v 0) v1 -- 0x7f796804e170 con 0x7f79680b85d0 2023-10-16T08:18:32.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f795dffb640 1 -- 172.21.15.32:0/3777624934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]}]=0 updated caps for client.0 v12) v1 ==== 189+0+146 (secure 0 0 0) 0x7f79640ef310 con 0x7f79680b85d0 2023-10-16T08:18:32.457 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:18:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:18:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-10-16T08:18:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:18:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw pool=testcephfs1_data" 2023-10-16T08:18:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:18:32.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f794c0664a0 msgr2=0x7f794c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:32.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f794c0664a0 0x7f794c068960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f796815e490 tx=0x7f7958009190 comp rx=0 tx=0).stop 2023-10-16T08:18:32.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 msgr2=0x7f79680c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:32.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 0x7f79680c9d30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f79640ad040 tx=0x7f796401ac90 comp rx=0 tx=0).stop 2023-10-16T08:18:32.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 shutdown_connections 2023-10-16T08:18:32.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79680d0130 0x7f79680b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f794c0664a0 0x7f794c068960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f79680b89b0 0x7f796814fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 --2- 172.21.15.32:0/3777624934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79680b85d0 0x7f79680c9d30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:32.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 >> 172.21.15.32:0/3777624934 conn(0x7f796806b750 msgr2=0x7f796815d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:32.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 shutdown_connections 2023-10-16T08:18:32.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:32.453+0000 7f796f53e640 1 -- 172.21.15.32:0/3777624934 wait complete. 2023-10-16T08:18:32.479 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T08:18:32.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:18:32.592 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:18:32.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T08:18:32.675 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:32.676 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:32.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:18:32.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:18:32.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:32.686 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:18:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-16T08:18:32.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:18:32.743 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:18:32.744 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:32.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:18:32.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:18:32.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:32.755 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T08:18:32.807 INFO:teuthology.orchestra.run.smithi186.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-16T08:18:32.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:18:32.808 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:32.809 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:32.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:18:32.818 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:18:32.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:32.819 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T08:18:32.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:18:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-16T08:18:32.871 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:18:32.871 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:32.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:18:32.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:18:32.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:32.881 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T08:18:32.934 INFO:teuthology.orchestra.run.smithi186.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-16T08:18:32.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:18:32.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.073+0000 7fb1b5172640 1 Processor -- start 2023-10-16T08:18:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.073+0000 7fb1b5172640 1 -- start start 2023-10-16T08:18:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.073+0000 7fb1b5172640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb1b01597c0 0x7fb1b00ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1b5172640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b00adc10 0x7fb1b00cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1b5172640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b00723d0 con 0x7fb1b0159ba0 2023-10-16T08:18:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1b5172640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b00717c0 con 0x7fb1b00adc10 2023-10-16T08:18:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1b5172640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b0071aa0 con 0x7fb1b01597c0 2023-10-16T08:18:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1af59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b00adc10 0x7fb1b00cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1ae59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb1b0159ba0 0x7fb1b00cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49236/0 (socket says 172.21.15.32:49236) 2023-10-16T08:18:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1ae59c640 1 -- 172.21.15.32:0/1351564520 learned_addr learned my addr 172.21.15.32:0/1351564520 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1aed9d640 1 --2- 172.21.15.32:0/1351564520 >> v2:172.21.15.186:3300/0 conn(0x7fb1b01597c0 0x7fb1b00ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1af59e640 1 -- 172.21.15.32:0/1351564520 >> v2:172.21.15.186:3300/0 conn(0x7fb1b01597c0 msgr2=0x7fb1b00ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1af59e640 1 --2- 172.21.15.32:0/1351564520 >> v2:172.21.15.186:3300/0 conn(0x7fb1b01597c0 0x7fb1b00ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1af59e640 1 -- 172.21.15.32:0/1351564520 >> v1:172.21.15.32:6789/0 conn(0x7fb1b0159ba0 legacy=0x7fb1b00cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1af59e640 1 -- 172.21.15.32:0/1351564520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0071d80 con 0x7fb1b00adc10 2023-10-16T08:18:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1add9b640 1 -- 172.21.15.32:0/1351564520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644395366 0 0) 0x7fb1b00723d0 con 0x7fb1b0159ba0 2023-10-16T08:18:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1af59e640 1 --2- 172.21.15.32:0/1351564520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00adc10 0x7fb1b00cb520 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00cdaf0 tx=0x7fb1a801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8848e1c23099df6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1add9b640 1 -- 172.21.15.32:0/1351564520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80a5970 con 0x7fb1b00adc10 2023-10-16T08:18:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1add9b640 1 -- 172.21.15.32:0/1351564520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a80a5ad0 con 0x7fb1b00adc10 2023-10-16T08:18:33.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.077+0000 7fb1add9b640 1 -- 172.21.15.32:0/1351564520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80a5e60 con 0x7fb1b00adc10 2023-10-16T08:18:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- 172.21.15.32:0/1351564520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00adc10 msgr2=0x7fb1b00cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 --2- 172.21.15.32:0/1351564520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00adc10 0x7fb1b00cb520 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00cdaf0 tx=0x7fb1a801c910 comp rx=0 tx=0).stop 2023-10-16T08:18:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- 172.21.15.32:0/1351564520 shutdown_connections 2023-10-16T08:18:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 --2- 172.21.15.32:0/1351564520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00adc10 0x7fb1b00cb520 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 --2- 172.21.15.32:0/1351564520 >> v2:172.21.15.186:3300/0 conn(0x7fb1b01597c0 0x7fb1b00ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- 172.21.15.32:0/1351564520 >> 172.21.15.32:0/1351564520 conn(0x7fb1b006c190 msgr2=0x7fb1b006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- 172.21.15.32:0/1351564520 shutdown_connections 2023-10-16T08:18:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- 172.21.15.32:0/1351564520 wait complete. 2023-10-16T08:18:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 Processor -- start 2023-10-16T08:18:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- start start 2023-10-16T08:18:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1b00adc10 0x7fb1b0159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1b01597c0 0x7fb1b014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 0x7fb1b0149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0154250 con 0x7fb1b0159ba0 2023-10-16T08:18:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0154530 con 0x7fb1b00adc10 2023-10-16T08:18:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1b5172640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0154810 con 0x7fb1b01597c0 2023-10-16T08:18:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 0x7fb1b0149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 0x7fb1b0149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56636/0 (socket says 172.21.15.32:56636) 2023-10-16T08:18:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1af59e640 1 -- 172.21.15.32:0/194915907 learned_addr learned my addr 172.21.15.32:0/194915907 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1aed9d640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1b00adc10 0x7fb1b0159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.081+0000 7fb1ae59c640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1b01597c0 0x7fb1b014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1af59e640 1 -- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1b01597c0 msgr2=0x7fb1b014ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1af59e640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1b01597c0 0x7fb1b014ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1af59e640 1 -- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1b00adc10 msgr2=0x7fb1b0159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1af59e640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1b00adc10 0x7fb1b0159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1af59e640 1 -- 172.21.15.32:0/194915907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009a70 con 0x7fb1b0159ba0 2023-10-16T08:18:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1af59e640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 0x7fb1b0149360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80b0040 tx=0x7fb1a80196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80adcc0 con 0x7fb1b0159ba0 2023-10-16T08:18:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a80ade20 con 0x7fb1b0159ba0 2023-10-16T08:18:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80b9690 con 0x7fb1b0159ba0 2023-10-16T08:18:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b0151e60 con 0x7fb1b0159ba0 2023-10-16T08:18:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b0151b10 con 0x7fb1b0159ba0 2023-10-16T08:18:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b0074010 con 0x7fb1b0159ba0 2023-10-16T08:18:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb1a80b97f0 con 0x7fb1b0159ba0 2023-10-16T08:18:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1977fe640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb19c0664a0 0x7fb19c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.085+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fb1a811f650 con 0x7fb1b0159ba0 2023-10-16T08:18:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.089+0000 7fb1aed9d640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb19c0664a0 0x7fb19c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.089+0000 7fb1aed9d640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb19c0664a0 0x7fb19c068960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009b60 tx=0x7fb1a0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.089+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb1a80ec880 con 0x7fb1b0159ba0 2023-10-16T08:18:33.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.277+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1b0152310 con 0x7fb1b0159ba0 2023-10-16T08:18:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1977fe640 1 -- 172.21.15.32:0/194915907 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fb1a8009450 con 0x7fb1b0159ba0 2023-10-16T08:18:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:33.286 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:18:01.158878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:18:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2023-10-16T08:18:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb19c0664a0 msgr2=0x7fb19c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb19c0664a0 0x7fb19c068960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009b60 tx=0x7fb1a0009190 comp rx=0 tx=0).stop 2023-10-16T08:18:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 msgr2=0x7fb1b0149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 0x7fb1b0149360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80b0040 tx=0x7fb1a80196b0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 shutdown_connections 2023-10-16T08:18:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b0159ba0 0x7fb1b0149360 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb19c0664a0 0x7fb19c068960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb1b01597c0 0x7fb1b014ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 --2- 172.21.15.32:0/194915907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1b00adc10 0x7fb1b0159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 >> 172.21.15.32:0/194915907 conn(0x7fb1b006c190 msgr2=0x7fb1b00ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 shutdown_connections 2023-10-16T08:18:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.281+0000 7fb1b5172640 1 -- 172.21.15.32:0/194915907 wait complete. 2023-10-16T08:18:33.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:33.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 Processor -- start 2023-10-16T08:18:33.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 -- start start 2023-10-16T08:18:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2fd815cb90 0x7f2fd80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fd80d1400 0x7f2fd80cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd80d1890 con 0x7f2fd80d1400 2023-10-16T08:18:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd80ce770 con 0x7f2fd815cb90 2023-10-16T08:18:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fdcf0f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd80d0600 con 0x7f2fd80d3010 2023-10-16T08:18:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fd80d1400 0x7f2fd80cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fd80d1400 0x7f2fd80cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56646/0 (socket says 172.21.15.32:56646) 2023-10-16T08:18:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1819217344 learned_addr learned my addr 172.21.15.32:0/1819217344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fcffff640 1 --2- 172.21.15.32:0/1819217344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd80d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd659c640 1 --2- 172.21.15.32:0/1819217344 >> v2:172.21.15.186:3300/0 conn(0x7f2fd815cb90 0x7f2fd80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1819217344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 msgr2=0x7f2fd80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 --2- 172.21.15.32:0/1819217344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd80d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1819217344 >> v2:172.21.15.186:3300/0 conn(0x7f2fd815cb90 msgr2=0x7f2fd80cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:33.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 --2- 172.21.15.32:0/1819217344 >> v2:172.21.15.186:3300/0 conn(0x7f2fd815cb90 0x7f2fd80cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.445+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1819217344 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80c6450 con 0x7f2fd80d1400 2023-10-16T08:18:33.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fd6d9d640 1 --2- 172.21.15.32:0/1819217344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd80d1400 0x7f2fd80cd3d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2fd808ba70 tx=0x7f2fd001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5ef2be95ad06a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fd5d9b640 1 -- 172.21.15.32:0/1819217344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00a5790 con 0x7f2fd80d1400 2023-10-16T08:18:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fd5d9b640 1 -- 172.21.15.32:0/1819217344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd00a58f0 con 0x7f2fd80d1400 2023-10-16T08:18:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fd5d9b640 1 -- 172.21.15.32:0/1819217344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00a5c80 con 0x7f2fd80d1400 2023-10-16T08:18:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1819217344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd80d1400 msgr2=0x7f2fd80cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1819217344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd80d1400 0x7f2fd80cd3d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2fd808ba70 tx=0x7f2fd001c910 comp rx=0 tx=0).stop 2023-10-16T08:18:33.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1819217344 shutdown_connections 2023-10-16T08:18:33.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1819217344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd80d1400 0x7f2fd80cd3d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1819217344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd80d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1819217344 >> v2:172.21.15.186:3300/0 conn(0x7f2fd815cb90 0x7f2fd80cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1819217344 >> 172.21.15.32:0/1819217344 conn(0x7f2fd809bc50 msgr2=0x7f2fd80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:33.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1819217344 shutdown_connections 2023-10-16T08:18:33.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.449+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1819217344 wait complete. 2023-10-16T08:18:33.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 Processor -- start 2023-10-16T08:18:33.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 -- start start 2023-10-16T08:18:33.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd80d1400 0x7f2fd814b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd81498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 0x7f2fd80b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8158680 con 0x7f2fd815cb90 2023-10-16T08:18:33.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815b580 con 0x7f2fd80d1400 2023-10-16T08:18:33.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81583a0 con 0x7f2fd80d3010 2023-10-16T08:18:33.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 0x7f2fd80b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 0x7f2fd80b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56660/0 (socket says 172.21.15.32:56660) 2023-10-16T08:18:33.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1717684931 learned_addr learned my addr 172.21.15.32:0/1717684931 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd659c640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd80d1400 0x7f2fd814b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fcffff640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd81498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 msgr2=0x7f2fd81498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd81498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd80d1400 msgr2=0x7f2fd814b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd80d1400 0x7f2fd814b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 -- 172.21.15.32:0/1717684931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009b30 con 0x7f2fd815cb90 2023-10-16T08:18:33.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd659c640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd80d1400 0x7f2fd814b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:33.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fd6d9d640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 0x7f2fd80b7470 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fd001afc0 tx=0x7f2fd00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00b8670 con 0x7f2fd815cb90 2023-10-16T08:18:33.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd00b87f0 con 0x7f2fd815cb90 2023-10-16T08:18:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd815af30 con 0x7f2fd815cb90 2023-10-16T08:18:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.453+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00b8ba0 con 0x7f2fd815cb90 2023-10-16T08:18:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd815ba10 con 0x7f2fd815cb90 2023-10-16T08:18:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd815ad90 con 0x7f2fd815cb90 2023-10-16T08:18:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2fd00b8d00 con 0x7f2fd815cb90 2023-10-16T08:18:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fceffd640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2fb80664d0 0x7f2fb8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f2fd011f110 con 0x7f2fd815cb90 2023-10-16T08:18:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fd659c640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2fb80664d0 0x7f2fb8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.457+0000 7f2fd659c640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2fb80664d0 0x7f2fb8068990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009a70 tx=0x7f2fc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.461+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2fd815ad90 con 0x7f2fd815cb90 2023-10-16T08:18:33.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.617+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fd815ad90 con 0x7f2fd815cb90 2023-10-16T08:18:33.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fceffd640 1 -- 172.21.15.32:0/1717684931 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7f2fd815ad90 con 0x7f2fd815cb90 2023-10-16T08:18:33.627 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:33.627 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2fb80664d0 msgr2=0x7f2fb8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2fb80664d0 0x7f2fb8068990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009a70 tx=0x7f2fc8008040 comp rx=0 tx=0).stop 2023-10-16T08:18:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 msgr2=0x7f2fd80b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 0x7f2fd80b7470 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fd001afc0 tx=0x7f2fd00196b0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 shutdown_connections 2023-10-16T08:18:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd815cb90 0x7f2fd80b7470 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2fb80664d0 0x7f2fb8068990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2fd80d3010 0x7f2fd81498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 --2- 172.21.15.32:0/1717684931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fd80d1400 0x7f2fd814b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 >> 172.21.15.32:0/1717684931 conn(0x7f2fd809bc50 msgr2=0x7f2fd80b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:33.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 shutdown_connections 2023-10-16T08:18:33.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.621+0000 7f2fdcf0f640 1 -- 172.21.15.32:0/1717684931 wait complete. 2023-10-16T08:18:33.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:17:49.375624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:17:50.533974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-16T08:18:00.037246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:33.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 Processor -- start 2023-10-16T08:18:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 -- start start 2023-10-16T08:18:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f41040d0210 0x7f41040d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41040b84e0 0x7f41040b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41040d1670 con 0x7f410415cba0 2023-10-16T08:18:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41040d1460 con 0x7f41040b84e0 2023-10-16T08:18:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4108e6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41040d1950 con 0x7f41040d0210 2023-10-16T08:18:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f410259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f410415cba0 0x7f41040cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49242/0 (socket says 172.21.15.32:49242) 2023-10-16T08:18:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f410259c640 1 -- 172.21.15.32:0/2485392722 learned_addr learned my addr 172.21.15.32:0/2485392722 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4102d9d640 1 --2- 172.21.15.32:0/2485392722 >> v2:172.21.15.32:3300/0 conn(0x7f41040b84e0 0x7f41040b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4101d9b640 1 --2- 172.21.15.32:0/2485392722 >> v2:172.21.15.186:3300/0 conn(0x7f41040d0210 0x7f41040d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4102d9d640 1 -- 172.21.15.32:0/2485392722 >> v2:172.21.15.186:3300/0 conn(0x7f41040d0210 msgr2=0x7f41040d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4102d9d640 1 --2- 172.21.15.32:0/2485392722 >> v2:172.21.15.186:3300/0 conn(0x7f41040d0210 0x7f41040d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4102d9d640 1 -- 172.21.15.32:0/2485392722 >> v1:172.21.15.32:6789/0 conn(0x7f410415cba0 legacy=0x7f41040cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f410159a640 1 -- 172.21.15.32:0/2485392722 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708030605 0 0) 0x7f41040d1670 con 0x7f410415cba0 2023-10-16T08:18:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4102d9d640 1 -- 172.21.15.32:0/2485392722 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41040d1c30 con 0x7f41040b84e0 2023-10-16T08:18:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.793+0000 7f4101d9b640 1 --2- 172.21.15.32:0/2485392722 >> v2:172.21.15.186:3300/0 conn(0x7f41040d0210 0x7f41040d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4102d9d640 1 --2- 172.21.15.32:0/2485392722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f41040b5cf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f41040cf010 tx=0x7f40fc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd890107e54d3bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f410159a640 1 -- 172.21.15.32:0/2485392722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0a5940 con 0x7f41040b84e0 2023-10-16T08:18:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f410159a640 1 -- 172.21.15.32:0/2485392722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc0a5aa0 con 0x7f41040b84e0 2023-10-16T08:18:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f410159a640 1 -- 172.21.15.32:0/2485392722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0a5e30 con 0x7f41040b84e0 2023-10-16T08:18:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 -- 172.21.15.32:0/2485392722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 msgr2=0x7f41040b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 --2- 172.21.15.32:0/2485392722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f41040b5cf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f41040cf010 tx=0x7f40fc01c880 comp rx=0 tx=0).stop 2023-10-16T08:18:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 -- 172.21.15.32:0/2485392722 shutdown_connections 2023-10-16T08:18:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 --2- 172.21.15.32:0/2485392722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f41040b5cf0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 --2- 172.21.15.32:0/2485392722 >> v2:172.21.15.186:3300/0 conn(0x7f41040d0210 0x7f41040d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 -- 172.21.15.32:0/2485392722 >> 172.21.15.32:0/2485392722 conn(0x7f41040719c0 msgr2=0x7f4104071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 -- 172.21.15.32:0/2485392722 shutdown_connections 2023-10-16T08:18:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.797+0000 7f4108e6b640 1 -- 172.21.15.32:0/2485392722 wait complete. 2023-10-16T08:18:33.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 Processor -- start 2023-10-16T08:18:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 -- start start 2023-10-16T08:18:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f4104157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41040d0210 0x7f410414d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f410415cba0 0x7f410414e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4104158ed0 con 0x7f41040b84e0 2023-10-16T08:18:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41040cbf40 con 0x7f410415cba0 2023-10-16T08:18:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41041582c0 con 0x7f41040d0210 2023-10-16T08:18:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f4104157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f4104157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56686/0 (socket says 172.21.15.32:56686) 2023-10-16T08:18:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 -- 172.21.15.32:0/330542434 learned_addr learned my addr 172.21.15.32:0/330542434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4101d9b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41040d0210 0x7f410414d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4102d9d640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f410415cba0 0x7f410414e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 -- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41040d0210 msgr2=0x7f410414d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41040d0210 0x7f410414d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 -- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f410415cba0 msgr2=0x7f410414e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f410415cba0 0x7f410414e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 -- 172.21.15.32:0/330542434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc009a70 con 0x7f41040b84e0 2023-10-16T08:18:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f410259c640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f4104157c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c480 tx=0x7f40f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40074e0 con 0x7f41040b84e0 2023-10-16T08:18:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f4007660 con 0x7f41040b84e0 2023-10-16T08:18:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41041585a0 con 0x7f41040b84e0 2023-10-16T08:18:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4004a30 con 0x7f41040b84e0 2023-10-16T08:18:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.801+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4104158b60 con 0x7f41040b84e0 2023-10-16T08:18:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.805+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4104159080 con 0x7f41040b84e0 2023-10-16T08:18:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.805+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f40f4004b90 con 0x7f41040b84e0 2023-10-16T08:18:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.805+0000 7f40f2ffd640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f40e40664a0 0x7f40e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.805+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f40f4083660 con 0x7f41040b84e0 2023-10-16T08:18:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.805+0000 7f4101d9b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f40e40664a0 0x7f40e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.805+0000 7f4101d9b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f40e40664a0 0x7f40e4068960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f41040ccc20 tx=0x7f40f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.809+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4104159080 con 0x7f41040b84e0 2023-10-16T08:18:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f410404e170 con 0x7f41040b84e0 2023-10-16T08:18:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f40f2ffd640 1 -- 172.21.15.32:0/330542434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40f4054320 con 0x7f41040b84e0 2023-10-16T08:18:33.975 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f40e40664a0 msgr2=0x7f40e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f40e40664a0 0x7f40e4068960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f41040ccc20 tx=0x7f40f8008040 comp rx=0 tx=0).stop 2023-10-16T08:18:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 msgr2=0x7f4104157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f4104157c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c480 tx=0x7f40f400c940 comp rx=0 tx=0).stop 2023-10-16T08:18:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 shutdown_connections 2023-10-16T08:18:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f410415cba0 0x7f410414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f40e40664a0 0x7f40e4068960 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f41040d0210 0x7f410414d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 --2- 172.21.15.32:0/330542434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b84e0 0x7f4104157c30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 >> 172.21.15.32:0/330542434 conn(0x7f41040719c0 msgr2=0x7f410415b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 shutdown_connections 2023-10-16T08:18:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:33.969+0000 7f4108e6b640 1 -- 172.21.15.32:0/330542434 wait complete. 2023-10-16T08:18:33.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 Processor -- start 2023-10-16T08:18:34.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 -- start start 2023-10-16T08:18:34.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecf806d010 0x7fecf80c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf80c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf815ae30 con 0x7fecf8069e90 2023-10-16T08:18:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80cb090 con 0x7fecf806d010 2023-10-16T08:18:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.137+0000 7fecfe8ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf806d470 con 0x7fecf80c26b0 2023-10-16T08:18:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fecf8069e90 0x7fecf806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49258/0 (socket says 172.21.15.32:49258) 2023-10-16T08:18:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf7fff640 1 -- 172.21.15.32:0/152454043 learned_addr learned my addr 172.21.15.32:0/152454043 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf77fe640 1 --2- 172.21.15.32:0/152454043 >> v2:172.21.15.32:3300/0 conn(0x7fecf806d010 0x7fecf80c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecfce87640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf80c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf6ffd640 1 -- 172.21.15.32:0/152454043 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158521762 0 0) 0x7fecf815ae30 con 0x7fecf8069e90 2023-10-16T08:18:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf6ffd640 1 -- 172.21.15.32:0/152454043 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece0003530 con 0x7fecf8069e90 2023-10-16T08:18:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf77fe640 1 -- 172.21.15.32:0/152454043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 msgr2=0x7fecf80c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf77fe640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf80c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf77fe640 1 -- 172.21.15.32:0/152454043 >> v1:172.21.15.32:6789/0 conn(0x7fecf8069e90 legacy=0x7fecf806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf77fe640 1 -- 172.21.15.32:0/152454043 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf80ca510 con 0x7fecf806d010 2023-10-16T08:18:34.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecfce87640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf80c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:34.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf77fe640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf80c9ce0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fecec0097c0 tx=0x7fecec01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74620d5bd1576fc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf6ffd640 1 -- 172.21.15.32:0/152454043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0a8970 con 0x7fecf806d010 2023-10-16T08:18:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf6ffd640 1 -- 172.21.15.32:0/152454043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec0a8ad0 con 0x7fecf806d010 2023-10-16T08:18:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.141+0000 7fecf6ffd640 1 -- 172.21.15.32:0/152454043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0a8e60 con 0x7fecf806d010 2023-10-16T08:18:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- 172.21.15.32:0/152454043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 msgr2=0x7fecf80c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf80c9ce0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fecec0097c0 tx=0x7fecec01ec70 comp rx=0 tx=0).stop 2023-10-16T08:18:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- 172.21.15.32:0/152454043 shutdown_connections 2023-10-16T08:18:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf80c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/152454043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf80c9ce0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- 172.21.15.32:0/152454043 >> 172.21.15.32:0/152454043 conn(0x7fecf80b0da0 msgr2=0x7fecf80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- 172.21.15.32:0/152454043 shutdown_connections 2023-10-16T08:18:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- 172.21.15.32:0/152454043 wait complete. 2023-10-16T08:18:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 Processor -- start 2023-10-16T08:18:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- start start 2023-10-16T08:18:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecf8069e90 0x7fecf81ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815c790 con 0x7fecf806d010 2023-10-16T08:18:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815bb80 con 0x7fecf8069e90 2023-10-16T08:18:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecfe8ea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815be60 con 0x7fecf80c26b0 2023-10-16T08:18:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecf77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecf77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf815b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56700/0 (socket says 172.21.15.32:56700) 2023-10-16T08:18:34.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecf77fe640 1 -- 172.21.15.32:0/1128170081 learned_addr learned my addr 172.21.15.32:0/1128170081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:34.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.145+0000 7fecf7fff640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecf8069e90 0x7fecf81ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecfce87640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf77fe640 1 -- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 msgr2=0x7fecf815b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf77fe640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf815b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf77fe640 1 -- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecf8069e90 msgr2=0x7fecf81ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf77fe640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecf8069e90 0x7fecf81ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf77fe640 1 -- 172.21.15.32:0/1128170081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec009420 con 0x7fecf806d010 2023-10-16T08:18:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecfce87640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf7fff640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecf8069e90 0x7fecf81ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf77fe640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf815b010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b3040 tx=0x7fecec01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0b9630 con 0x7fecf806d010 2023-10-16T08:18:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec0b9790 con 0x7fecf806d010 2023-10-16T08:18:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0b9b40 con 0x7fecf806d010 2023-10-16T08:18:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf815c140 con 0x7fecf806d010 2023-10-16T08:18:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf8156050 con 0x7fecf806d010 2023-10-16T08:18:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf815c900 con 0x7fecf806d010 2023-10-16T08:18:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fecec0b7040 con 0x7fecf806d010 2023-10-16T08:18:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf4ff9640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fece40664a0 0x7fece4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fecec1220a0 con 0x7fecf806d010 2023-10-16T08:18:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.149+0000 7fecf7fff640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fece40664a0 0x7fece4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.153+0000 7fecf7fff640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fece40664a0 0x7fece4068960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fece80011f0 tx=0x7fece8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.153+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fecec0ef340 con 0x7fecf806d010 2023-10-16T08:18:34.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.313+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecf80caaa0 con 0x7fecf806d010 2023-10-16T08:18:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.313+0000 7fecf4ff9640 1 -- 172.21.15.32:0/1128170081 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7fecec0b50b0 con 0x7fecf806d010 2023-10-16T08:18:34.320 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:34.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fece40664a0 msgr2=0x7fece4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fece40664a0 0x7fece4068960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fece80011f0 tx=0x7fece8009040 comp rx=0 tx=0).stop 2023-10-16T08:18:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 msgr2=0x7fecf815b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf815b010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b3040 tx=0x7fecec01ba10 comp rx=0 tx=0).stop 2023-10-16T08:18:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 shutdown_connections 2023-10-16T08:18:34.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fece40664a0 0x7fece4068960 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fecf80c26b0 0x7fecf815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecf806d010 0x7fecf815b010 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 --2- 172.21.15.32:0/1128170081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecf8069e90 0x7fecf81ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 >> 172.21.15.32:0/1128170081 conn(0x7fecf80b0da0 msgr2=0x7fecf80c2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:34.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 shutdown_connections 2023-10-16T08:18:34.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.317+0000 7fecfe8ea640 1 -- 172.21.15.32:0/1128170081 wait complete. 2023-10-16T08:18:34.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 Processor -- start 2023-10-16T08:18:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 -- start start 2023-10-16T08:18:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc161390 0x7f71dc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71dc06e2f0 0x7f71dc06e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc0cdc30 con 0x7f71dc15ce10 2023-10-16T08:18:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc1617f0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71e1b9d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc06ecc0 con 0x7f71dc161390 2023-10-16T08:18:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.481+0000 7f71dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71dc06e2f0 0x7f71dc06e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71dc06e2f0 0x7f71dc06e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56704/0 (socket says 172.21.15.32:56704) 2023-10-16T08:18:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 -- 172.21.15.32:0/2441128933 learned_addr learned my addr 172.21.15.32:0/2441128933 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:34.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71daffd640 1 --2- 172.21.15.32:0/2441128933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc161390 0x7f71dc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 -- 172.21.15.32:0/2441128933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc161390 msgr2=0x7f71dc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 --2- 172.21.15.32:0/2441128933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc161390 0x7f71dc0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 -- 172.21.15.32:0/2441128933 >> v1:172.21.15.186:6789/0 conn(0x7f71dc15ce10 legacy=0x7f71dc0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 -- 172.21.15.32:0/2441128933 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc0c9750 con 0x7f71dc06e2f0 2023-10-16T08:18:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71dbfff640 1 --2- 172.21.15.32:0/2441128933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc06e780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009e50 tx=0x7f71d401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798ae374b48c63bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71da7fc640 1 -- 172.21.15.32:0/2441128933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40a58e0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71da7fc640 1 -- 172.21.15.32:0/2441128933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d40a5a40 con 0x7f71dc06e2f0 2023-10-16T08:18:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.485+0000 7f71da7fc640 1 -- 172.21.15.32:0/2441128933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40a5dd0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- 172.21.15.32:0/2441128933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 msgr2=0x7f71dc06e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/2441128933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc06e780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009e50 tx=0x7f71d401ca10 comp rx=0 tx=0).stop 2023-10-16T08:18:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- 172.21.15.32:0/2441128933 shutdown_connections 2023-10-16T08:18:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/2441128933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc06e780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/2441128933 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc161390 0x7f71dc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- 172.21.15.32:0/2441128933 >> 172.21.15.32:0/2441128933 conn(0x7f71dc0cef20 msgr2=0x7f71dc0cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- 172.21.15.32:0/2441128933 shutdown_connections 2023-10-16T08:18:34.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- 172.21.15.32:0/2441128933 wait complete. 2023-10-16T08:18:34.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 Processor -- start 2023-10-16T08:18:34.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- start start 2023-10-16T08:18:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71dc161390 0x7f71dc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc15a1f0 0x7f71dc156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc15a720 con 0x7f71dc06e2f0 2023-10-16T08:18:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc15aa00 con 0x7f71dc161390 2023-10-16T08:18:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71e1b9d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc15ace0 con 0x7f71dc15a1f0 2023-10-16T08:18:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc15b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56718/0 (socket says 172.21.15.32:56718) 2023-10-16T08:18:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 -- 172.21.15.32:0/822549506 learned_addr learned my addr 172.21.15.32:0/822549506 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:34.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71dbfff640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc15a1f0 0x7f71dc156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71daffd640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71dc161390 0x7f71dc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 -- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc15a1f0 msgr2=0x7f71dc156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc15a1f0 0x7f71dc156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 -- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71dc161390 msgr2=0x7f71dc155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71dc161390 0x7f71dc155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.489+0000 7f71db7fe640 1 -- 172.21.15.32:0/822549506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4009b30 con 0x7f71dc06e2f0 2023-10-16T08:18:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71db7fe640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc15b7c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c420 tx=0x7f71cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0052d0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc005430 con 0x7f71dc06e2f0 2023-10-16T08:18:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc15afc0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc007260 con 0x7f71dc06e2f0 2023-10-16T08:18:34.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc15bd00 con 0x7f71dc06e2f0 2023-10-16T08:18:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc04e170 con 0x7f71dc06e2f0 2023-10-16T08:18:34.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f71cc0073c0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71c3fff640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71c80664d0 0x7f71c8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f71cc083490 con 0x7f71dc06e2f0 2023-10-16T08:18:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.493+0000 7f71daffd640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71c80664d0 0x7f71c8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.497+0000 7f71daffd640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71c80664d0 0x7f71c8068990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0c9350 tx=0x7f71d0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.497+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f71cc0506c0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.653+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f71dc06ef60 con 0x7f71dc06e2f0 2023-10-16T08:18:34.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.653+0000 7f71c3fff640 1 -- 172.21.15.32:0/822549506 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v42) v1 ==== 105+0+11 (secure 0 0 0) 0x7f71cc053ce0 con 0x7f71dc06e2f0 2023-10-16T08:18:34.661 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:34.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71c80664d0 msgr2=0x7f71c8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71c80664d0 0x7f71c8068990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0c9350 tx=0x7f71d0009190 comp rx=0 tx=0).stop 2023-10-16T08:18:34.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 msgr2=0x7f71dc15b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc15b7c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c420 tx=0x7f71cc00c8e0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 shutdown_connections 2023-10-16T08:18:34.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f71c80664d0 0x7f71c8068990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f71dc15a1f0 0x7f71dc156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71dc161390 0x7f71dc155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 --2- 172.21.15.32:0/822549506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71dc06e2f0 0x7f71dc15b7c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 >> 172.21.15.32:0/822549506 conn(0x7f71dc0cef20 msgr2=0x7f71dc15f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:34.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 shutdown_connections 2023-10-16T08:18:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.657+0000 7f71e1b9d640 1 -- 172.21.15.32:0/822549506 wait complete. 2023-10-16T08:18:34.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-16T08:18:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.821+0000 7fdc93017640 1 Processor -- start 2023-10-16T08:18:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc93017640 1 -- start start 2023-10-16T08:18:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc93017640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc8c0d0210 0x7fdc8c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc93017640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0d1670 con 0x7fdc8c15cba0 2023-10-16T08:18:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc93017640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0d1950 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc93017640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c0d1460 con 0x7fdc8c0d0210 2023-10-16T08:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc8bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc8c0d0210 0x7fdc8c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc90db3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc8c15cba0 0x7fdc8c0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49262/0 (socket says 172.21.15.32:49262) 2023-10-16T08:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc90db3640 1 -- 172.21.15.32:0/3628254180 learned_addr learned my addr 172.21.15.32:0/3628254180 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc8bfff640 1 -- 172.21.15.32:0/3628254180 >> v1:172.21.15.186:6789/0 conn(0x7fdc8c0b84e0 legacy=0x7fdc8c0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc8bfff640 1 -- 172.21.15.32:0/3628254180 >> v1:172.21.15.32:6789/0 conn(0x7fdc8c15cba0 legacy=0x7fdc8c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc8bfff640 1 -- 172.21.15.32:0/3628254180 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0d1c30 con 0x7fdc8c0d0210 2023-10-16T08:18:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.825+0000 7fdc8b7fe640 1 -- 172.21.15.32:0/3628254180 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579294931 0 0) 0x7fdc8c0d1670 con 0x7fdc8c15cba0 2023-10-16T08:18:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc8bfff640 1 --2- 172.21.15.32:0/3628254180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c0d0f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdc780097c0 tx=0x7fdc7801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3827a39c8461a27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc8b7fe640 1 -- 172.21.15.32:0/3628254180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780ab5f0 con 0x7fdc8c0d0210 2023-10-16T08:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc8b7fe640 1 -- 172.21.15.32:0/3628254180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc780ab750 con 0x7fdc8c0d0210 2023-10-16T08:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc8b7fe640 1 -- 172.21.15.32:0/3628254180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780abae0 con 0x7fdc8c0d0210 2023-10-16T08:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 -- 172.21.15.32:0/3628254180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0d0210 msgr2=0x7fdc8c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 --2- 172.21.15.32:0/3628254180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c0d0f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdc780097c0 tx=0x7fdc7801c3f0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 -- 172.21.15.32:0/3628254180 shutdown_connections 2023-10-16T08:18:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 --2- 172.21.15.32:0/3628254180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c0d0f20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 -- 172.21.15.32:0/3628254180 >> 172.21.15.32:0/3628254180 conn(0x7fdc8c0719c0 msgr2=0x7fdc8c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 -- 172.21.15.32:0/3628254180 shutdown_connections 2023-10-16T08:18:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 -- 172.21.15.32:0/3628254180 wait complete. 2023-10-16T08:18:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 Processor -- start 2023-10-16T08:18:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.829+0000 7fdc93017640 1 -- start start 2023-10-16T08:18:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 0x7fdc8c14f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c14f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc8c15cba0 0x7fdc8c0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c9520 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c14ffe0 con 0x7fdc8c0d0210 2023-10-16T08:18:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c14aaf0 con 0x7fdc8c15cba0 2023-10-16T08:18:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 0x7fdc8c14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 0x7fdc8c14f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56748/0 (socket says 172.21.15.32:56748) 2023-10-16T08:18:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 -- 172.21.15.32:0/323720596 learned_addr learned my addr 172.21.15.32:0/323720596 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc8bfff640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c14f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc915b4640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc8c15cba0 0x7fdc8c0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 -- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc8c15cba0 msgr2=0x7fdc8c0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc8c15cba0 0x7fdc8c0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 -- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc8c0d0210 msgr2=0x7fdc8c14f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c14f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 -- 172.21.15.32:0/323720596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009420 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc8bfff640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c14f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc90db3640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 0x7fdc8c14f030 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c410 tx=0x7fdc8000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc800074e0 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc80007640 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c14add0 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004a30 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c14b390 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c04e170 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.833+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc80004b90 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.837+0000 7fdc897fa640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdc700664a0 0x7fdc70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.837+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fdc80083f80 con 0x7fdc8c0b84e0 2023-10-16T08:18:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.837+0000 7fdc8bfff640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdc700664a0 0x7fdc70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:34.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.837+0000 7fdc8bfff640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdc700664a0 0x7fdc70068960 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc780af000 tx=0x7fdc780a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:34.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.841+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdc80051100 con 0x7fdc8c0b84e0 2023-10-16T08:18:35.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.997+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fdc8c0d3200 con 0x7fdc8c0b84e0 2023-10-16T08:18:35.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:34.997+0000 7fdc897fa640 1 -- 172.21.15.32:0/323720596 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7fdc80054720 con 0x7fdc8c0b84e0 2023-10-16T08:18:35.004 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:35.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdc700664a0 msgr2=0x7fdc70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdc700664a0 0x7fdc70068960 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc780af000 tx=0x7fdc780a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 msgr2=0x7fdc8c14f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 0x7fdc8c14f030 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c410 tx=0x7fdc8000c8d0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 shutdown_connections 2023-10-16T08:18:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdc700664a0 0x7fdc70068960 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdc8c15cba0 0x7fdc8c0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc8c0d0210 0x7fdc8c14f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 --2- 172.21.15.32:0/323720596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc8c0b84e0 0x7fdc8c14f030 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 >> 172.21.15.32:0/323720596 conn(0x7fdc8c0719c0 msgr2=0x7fdc8c06b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 shutdown_connections 2023-10-16T08:18:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.001+0000 7fdc93017640 1 -- 172.21.15.32:0/323720596 wait complete. 2023-10-16T08:18:35.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.161+0000 7f68ac6f7640 1 Processor -- start 2023-10-16T08:18:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.161+0000 7f68ac6f7640 1 -- start start 2023-10-16T08:18:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68ac6f7640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f68a415ca50 0x7f68a406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68ac6f7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68a40b21c0 0x7f68a40b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68ac6f7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a41607e0 con 0x7f68a415ce30 2023-10-16T08:18:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68ac6f7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a40cb4a0 con 0x7f68a40b21c0 2023-10-16T08:18:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68ac6f7640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a40cf060 con 0x7f68a415ca50 2023-10-16T08:18:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68a40b21c0 0x7f68a40b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68a40b21c0 0x7f68a40b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56752/0 (socket says 172.21.15.32:56752) 2023-10-16T08:18:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 -- 172.21.15.32:0/2636212417 learned_addr learned my addr 172.21.15.32:0/2636212417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aa493640 1 --2- 172.21.15.32:0/2636212417 >> v2:172.21.15.186:3300/0 conn(0x7f68a415ca50 0x7f68a406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 -- 172.21.15.32:0/2636212417 >> v2:172.21.15.186:3300/0 conn(0x7f68a415ca50 msgr2=0x7f68a406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 --2- 172.21.15.32:0/2636212417 >> v2:172.21.15.186:3300/0 conn(0x7f68a415ca50 0x7f68a406c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 -- 172.21.15.32:0/2636212417 >> v1:172.21.15.186:6789/0 conn(0x7f68a415ce30 legacy=0x7f68a40cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.165+0000 7f68aac94640 1 -- 172.21.15.32:0/2636212417 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a40b2ae0 con 0x7f68a40b21c0 2023-10-16T08:18:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68aac94640 1 --2- 172.21.15.32:0/2636212417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40b25a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f68a40726c0 tx=0x7f689c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7797960b25a000 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68a9491640 1 -- 172.21.15.32:0/2636212417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c0a5820 con 0x7f68a40b21c0 2023-10-16T08:18:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68a9491640 1 -- 172.21.15.32:0/2636212417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c0a5980 con 0x7f68a40b21c0 2023-10-16T08:18:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68a9491640 1 -- 172.21.15.32:0/2636212417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c0a5d10 con 0x7f68a40b21c0 2023-10-16T08:18:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 -- 172.21.15.32:0/2636212417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 msgr2=0x7f68a40b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/2636212417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40b25a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f68a40726c0 tx=0x7f689c01c960 comp rx=0 tx=0).stop 2023-10-16T08:18:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 -- 172.21.15.32:0/2636212417 shutdown_connections 2023-10-16T08:18:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/2636212417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40b25a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/2636212417 >> v2:172.21.15.186:3300/0 conn(0x7f68a415ca50 0x7f68a406c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 -- 172.21.15.32:0/2636212417 >> 172.21.15.32:0/2636212417 conn(0x7f68a40cd1c0 msgr2=0x7f68a40cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 -- 172.21.15.32:0/2636212417 shutdown_connections 2023-10-16T08:18:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 -- 172.21.15.32:0/2636212417 wait complete. 2023-10-16T08:18:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 Processor -- start 2023-10-16T08:18:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.169+0000 7f68ac6f7640 1 -- start start 2023-10-16T08:18:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a415ca50 0x7f68a415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68a415ce30 0x7f68a415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40af7e0 con 0x7f68a40b21c0 2023-10-16T08:18:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40af220 con 0x7f68a415ca50 2023-10-16T08:18:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40af500 con 0x7f68a415ce30 2023-10-16T08:18:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56758/0 (socket says 172.21.15.32:56758) 2023-10-16T08:18:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 -- 172.21.15.32:0/977140118 learned_addr learned my addr 172.21.15.32:0/977140118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68a9c92640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a415ca50 0x7f68a415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aac94640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68a415ce30 0x7f68a415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 -- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68a415ce30 msgr2=0x7f68a415f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68a415ce30 0x7f68a415f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 -- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a415ca50 msgr2=0x7f68a415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a415ca50 0x7f68a415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 -- 172.21.15.32:0/977140118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c009b30 con 0x7f68a40b21c0 2023-10-16T08:18:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68aa493640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40c6b00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f68a000d800 tx=0x7f68a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a000b6c0 con 0x7f68a40b21c0 2023-10-16T08:18:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a000b820 con 0x7f68a40b21c0 2023-10-16T08:18:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a000bb90 con 0x7f68a40b21c0 2023-10-16T08:18:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a40c8b20 con 0x7f68a40b21c0 2023-10-16T08:18:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a40ae430 con 0x7f68a40b21c0 2023-10-16T08:18:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a4074010 con 0x7f68a40b21c0 2023-10-16T08:18:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68a000bcf0 con 0x7f68a40b21c0 2023-10-16T08:18:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.173+0000 7f689affd640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68880664a0 0x7f6888068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.177+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f68a00847a0 con 0x7f68a40b21c0 2023-10-16T08:18:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.177+0000 7f68a9c92640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68880664a0 0x7f6888068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.177+0000 7f68a9c92640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68880664a0 0x7f6888068960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68a41f8230 tx=0x7f6894008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.177+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68a0051890 con 0x7f68a40b21c0 2023-10-16T08:18:35.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.369+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68a40cddb0 con 0x7f68a40b21c0 2023-10-16T08:18:35.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.369+0000 7f689affd640 1 -- 172.21.15.32:0/977140118 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f68a0011340 con 0x7f68a40b21c0 2023-10-16T08:18:35.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:35.377 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:18:01.158878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:18:35.378 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2023-10-16T08:18:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68880664a0 msgr2=0x7f6888068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68880664a0 0x7f6888068960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68a41f8230 tx=0x7f6894008040 comp rx=0 tx=0).stop 2023-10-16T08:18:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 msgr2=0x7f68a40c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40c6b00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f68a000d800 tx=0x7f68a000dcc0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 shutdown_connections 2023-10-16T08:18:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f68880664a0 0x7f6888068960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f68a415ce30 0x7f68a415f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68a415ca50 0x7f68a415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 --2- 172.21.15.32:0/977140118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a40b21c0 0x7f68a40c6b00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 >> 172.21.15.32:0/977140118 conn(0x7f68a40cd1c0 msgr2=0x7f68a406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 shutdown_connections 2023-10-16T08:18:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.373+0000 7f68ac6f7640 1 -- 172.21.15.32:0/977140118 wait complete. 2023-10-16T08:18:35.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.533+0000 7f90df891640 1 Processor -- start 2023-10-16T08:18:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 -- start start 2023-10-16T08:18:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90d8070c40 0x7f90d815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f90d80b4020 0x7f90d80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d80b0620 0x7f90d815c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d815b3b0 con 0x7f90d8070c40 2023-10-16T08:18:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d80b0a70 con 0x7f90d80b4020 2023-10-16T08:18:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90df891640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d815be60 con 0x7f90d80b0620 2023-10-16T08:18:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90d8070c40 0x7f90d815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90d8070c40 0x7f90d815ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56770/0 (socket says 172.21.15.32:56770) 2023-10-16T08:18:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 -- 172.21.15.32:0/1571515263 learned_addr learned my addr 172.21.15.32:0/1571515263 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dde2e640 1 --2- 172.21.15.32:0/1571515263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d80b0620 0x7f90d815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dce2c640 1 --2- 172.21.15.32:0/1571515263 >> v2:172.21.15.186:3300/0 conn(0x7f90d80b4020 0x7f90d80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 -- 172.21.15.32:0/1571515263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d80b0620 msgr2=0x7f90d815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1571515263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d80b0620 0x7f90d815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 -- 172.21.15.32:0/1571515263 >> v2:172.21.15.186:3300/0 conn(0x7f90d80b4020 msgr2=0x7f90d80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1571515263 >> v2:172.21.15.186:3300/0 conn(0x7f90d80b4020 0x7f90d80b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.537+0000 7f90dd62d640 1 -- 172.21.15.32:0/1571515263 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d815c1f0 con 0x7f90d8070c40 2023-10-16T08:18:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1571515263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d8070c40 0x7f90d815ae70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90d80cc1e0 tx=0x7f90d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f98d35ef658712d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90cffff640 1 -- 172.21.15.32:0/1571515263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40a7070 con 0x7f90d8070c40 2023-10-16T08:18:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90cffff640 1 -- 172.21.15.32:0/1571515263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d40ab430 con 0x7f90d8070c40 2023-10-16T08:18:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90cffff640 1 -- 172.21.15.32:0/1571515263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40ab7c0 con 0x7f90d8070c40 2023-10-16T08:18:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 -- 172.21.15.32:0/1571515263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d8070c40 msgr2=0x7f90d815ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 --2- 172.21.15.32:0/1571515263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d8070c40 0x7f90d815ae70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90d80cc1e0 tx=0x7f90d401c470 comp rx=0 tx=0).stop 2023-10-16T08:18:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 -- 172.21.15.32:0/1571515263 shutdown_connections 2023-10-16T08:18:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 --2- 172.21.15.32:0/1571515263 >> v2:172.21.15.186:3300/0 conn(0x7f90d80b4020 0x7f90d80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 --2- 172.21.15.32:0/1571515263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d80b0620 0x7f90d815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 --2- 172.21.15.32:0/1571515263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d8070c40 0x7f90d815ae70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 -- 172.21.15.32:0/1571515263 >> 172.21.15.32:0/1571515263 conn(0x7f90d806c190 msgr2=0x7f90d806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 -- 172.21.15.32:0/1571515263 shutdown_connections 2023-10-16T08:18:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 -- 172.21.15.32:0/1571515263 wait complete. 2023-10-16T08:18:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 Processor -- start 2023-10-16T08:18:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.541+0000 7f90df891640 1 -- start start 2023-10-16T08:18:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d8070c40 0x7f90d80b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 0x7f90d814e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90d80b4020 0x7f90d80c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80b2d20 con 0x7f90d80b0620 2023-10-16T08:18:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d814ce90 con 0x7f90d80b4020 2023-10-16T08:18:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d814eb60 con 0x7f90d8070c40 2023-10-16T08:18:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 0x7f90d814e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 0x7f90d814e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56786/0 (socket says 172.21.15.32:56786) 2023-10-16T08:18:35.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 -- 172.21.15.32:0/1633864033 learned_addr learned my addr 172.21.15.32:0/1633864033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:35.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dde2e640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90d80b4020 0x7f90d80c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d8070c40 0x7f90d80b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 -- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d8070c40 msgr2=0x7f90d80b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d8070c40 0x7f90d80b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 -- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90d80b4020 msgr2=0x7f90d80c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90d80b4020 0x7f90d80c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 -- 172.21.15.32:0/1633864033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d40094d0 con 0x7f90d80b0620 2023-10-16T08:18:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d8070c40 0x7f90d80b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dde2e640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90d80b4020 0x7f90d80c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:35.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90dce2c640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 0x7f90d814e620 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f90c800d620 tx=0x7f90c800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800b620 con 0x7f90d80b0620 2023-10-16T08:18:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800b780 con 0x7f90d80b0620 2023-10-16T08:18:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800bb10 con 0x7f90d80b0620 2023-10-16T08:18:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d80b2760 con 0x7f90d80b0620 2023-10-16T08:18:35.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d814cb20 con 0x7f90d80b0620 2023-10-16T08:18:35.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d80b0450 con 0x7f90d80b0620 2023-10-16T08:18:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.545+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f90c800bc70 con 0x7f90d80b0620 2023-10-16T08:18:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.549+0000 7f90cdffb640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90bc0664a0 0x7f90bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.549+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f90c8084540 con 0x7f90d80b0620 2023-10-16T08:18:35.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.549+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90bc0664a0 0x7f90bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.549+0000 7f90dd62d640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90bc0664a0 0x7f90bc068960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f90d40af000 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.553+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f90c8051740 con 0x7f90d80b0620 2023-10-16T08:18:35.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90d8074010 con 0x7f90d80b0620 2023-10-16T08:18:35.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90cdffb640 1 -- 172.21.15.32:0/1633864033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7f90c8054d60 con 0x7f90d80b0620 2023-10-16T08:18:35.719 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:35.719 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90bc0664a0 msgr2=0x7f90bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90bc0664a0 0x7f90bc068960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f90d40af000 tx=0x7f90d4008040 comp rx=0 tx=0).stop 2023-10-16T08:18:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 msgr2=0x7f90d814e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 0x7f90d814e620 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f90c800d620 tx=0x7f90c800dae0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 shutdown_connections 2023-10-16T08:18:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90d80b4020 0x7f90d80c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d80b0620 0x7f90d814e620 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f90bc0664a0 0x7f90bc068960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 --2- 172.21.15.32:0/1633864033 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f90d8070c40 0x7f90d80b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 >> 172.21.15.32:0/1633864033 conn(0x7f90d806c190 msgr2=0x7f90d806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 shutdown_connections 2023-10-16T08:18:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.713+0000 7f90df891640 1 -- 172.21.15.32:0/1633864033 wait complete. 2023-10-16T08:18:35.740 INFO:tasks.cephfs.filesystem:Destroying file system testcephfs1 and related pools 2023-10-16T08:18:35.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:35.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.877+0000 7f9add674640 1 Processor -- start 2023-10-16T08:18:35.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.877+0000 7f9add674640 1 -- start start 2023-10-16T08:18:35.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9add674640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad815c980 0x7f9ad806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9add674640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80d3510 con 0x7f9ad815cd60 2023-10-16T08:18:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9add674640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80d2f50 con 0x7f9ad80d1490 2023-10-16T08:18:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9add674640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80cd2b0 con 0x7f9ad815c980 2023-10-16T08:18:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ad815cd60 0x7f9ad80d0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49268/0 (socket says 172.21.15.32:49268) 2023-10-16T08:18:35.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad77fe640 1 -- 172.21.15.32:0/266369693 learned_addr learned my addr 172.21.15.32:0/266369693 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad7fff640 1 --2- 172.21.15.32:0/266369693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad815c980 0x7f9ad806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230978510 0 0) 0x7f9acc001120 con 0x7f9ad815cd60 2023-10-16T08:18:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac0003530 con 0x7f9ad815cd60 2023-10-16T08:18:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204388233 0 0) 0x7f9ad80d2f50 con 0x7f9ad80d1490 2023-10-16T08:18:35.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc001120 con 0x7f9ad80d1490 2023-10-16T08:18:35.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad7fff640 1 -- 172.21.15.32:0/266369693 >> v1:172.21.15.186:6789/0 conn(0x7f9ad80d1490 legacy=0x7f9ad80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad7fff640 1 -- 172.21.15.32:0/266369693 >> v1:172.21.15.32:6789/0 conn(0x7f9ad815cd60 legacy=0x7f9ad80d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.881+0000 7f9ad7fff640 1 -- 172.21.15.32:0/266369693 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80d3230 con 0x7f9ad815c980 2023-10-16T08:18:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9ad7fff640 1 --2- 172.21.15.32:0/266369693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad815c980 0x7f9ad806af90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009800 tx=0x7f9ac801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e63696127a9bb82 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80a6040 con 0x7f9ad815c980 2023-10-16T08:18:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac80aa600 con 0x7f9ad815c980 2023-10-16T08:18:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9ad6ffd640 1 -- 172.21.15.32:0/266369693 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80aa970 con 0x7f9ad815c980 2023-10-16T08:18:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 -- 172.21.15.32:0/266369693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad815c980 msgr2=0x7f9ad806af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 --2- 172.21.15.32:0/266369693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad815c980 0x7f9ad806af90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009800 tx=0x7f9ac801c480 comp rx=0 tx=0).stop 2023-10-16T08:18:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 -- 172.21.15.32:0/266369693 shutdown_connections 2023-10-16T08:18:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 --2- 172.21.15.32:0/266369693 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad815c980 0x7f9ad806af90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 -- 172.21.15.32:0/266369693 >> 172.21.15.32:0/266369693 conn(0x7f9ad809ba80 msgr2=0x7f9ad80d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 -- 172.21.15.32:0/266369693 shutdown_connections 2023-10-16T08:18:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 -- 172.21.15.32:0/266369693 wait complete. 2023-10-16T08:18:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 Processor -- start 2023-10-16T08:18:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.885+0000 7f9add674640 1 -- start start 2023-10-16T08:18:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad80d1490 0x7f9ad80b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ad815c980 0x7f9ad815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 0x7f9ad80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad81f5b10 con 0x7f9ad815cd60 2023-10-16T08:18:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815ab30 con 0x7f9ad815c980 2023-10-16T08:18:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815ae10 con 0x7f9ad80d1490 2023-10-16T08:18:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 0x7f9ad80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 0x7f9ad80b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56802/0 (socket says 172.21.15.32:56802) 2023-10-16T08:18:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 -- 172.21.15.32:0/362119762 learned_addr learned my addr 172.21.15.32:0/362119762 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9ad7fff640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad80d1490 0x7f9ad80b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9ad77fe640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ad815c980 0x7f9ad815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 -- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad80d1490 msgr2=0x7f9ad80b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad80d1490 0x7f9ad80b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 -- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ad815c980 msgr2=0x7f9ad815a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ad815c980 0x7f9ad815a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 -- 172.21.15.32:0/362119762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80093b0 con 0x7f9ad815cd60 2023-10-16T08:18:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9adce73640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 0x7f9ad80b3d40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c790 tx=0x7f9ad000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000cea0 con 0x7f9ad815cd60 2023-10-16T08:18:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0003b80 con 0x7f9ad815cd60 2023-10-16T08:18:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000a850 con 0x7f9ad815cd60 2023-10-16T08:18:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad815b0f0 con 0x7f9ad815cd60 2023-10-16T08:18:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad815a7d0 con 0x7f9ad815cd60 2023-10-16T08:18:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad81f5cc0 con 0x7f9ad815cd60 2023-10-16T08:18:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.889+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9ad000a9b0 con 0x7f9ad815cd60 2023-10-16T08:18:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.893+0000 7f9ad4ff9640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ac40664a0 0x7f9ac4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.893+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f9ad0083ff0 con 0x7f9ad815cd60 2023-10-16T08:18:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.893+0000 7f9ad7fff640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ac40664a0 0x7f9ac4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.893+0000 7f9ad7fff640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ac40664a0 0x7f9ac4068960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80ae000 tx=0x7f9ac80b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:35.897+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9ad00b5090 con 0x7f9ad815cd60 2023-10-16T08:18:36.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ad80d0870 con 0x7f9ad815cd60 2023-10-16T08:18:36.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9ad4ff9640 1 -- 172.21.15.32:0/362119762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f9ad81f5cc0 con 0x7f9ad815cd60 2023-10-16T08:18:36.095 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:36.096 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:18:01.158878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:18:36.096 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2023-10-16T08:18:36.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ac40664a0 msgr2=0x7f9ac4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ac40664a0 0x7f9ac4068960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80ae000 tx=0x7f9ac80b5040 comp rx=0 tx=0).stop 2023-10-16T08:18:36.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 msgr2=0x7f9ad80b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 0x7f9ad80b3d40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c790 tx=0x7f9ad000cc50 comp rx=0 tx=0).stop 2023-10-16T08:18:36.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 shutdown_connections 2023-10-16T08:18:36.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad815cd60 0x7f9ad80b3d40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ad815c980 0x7f9ad815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9ac40664a0 0x7f9ac4068960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 --2- 172.21.15.32:0/362119762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9ad80d1490 0x7f9ad80b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 >> 172.21.15.32:0/362119762 conn(0x7f9ad809ba80 msgr2=0x7f9ad806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:36.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 shutdown_connections 2023-10-16T08:18:36.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.089+0000 7f9add674640 1 -- 172.21.15.32:0/362119762 wait complete. 2023-10-16T08:18:36.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.253+0000 7f6d9e405640 1 Processor -- start 2023-10-16T08:18:36.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.253+0000 7f6d9e405640 1 -- start start 2023-10-16T08:18:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9e405640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f6d98069e90 0x7f6d9806a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9e405640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d980ca5f0 0x7f6d980caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9e405640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d980ca150 con 0x7f6d9815cd10 2023-10-16T08:18:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9e405640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9806a880 con 0x7f6d980ca5f0 2023-10-16T08:18:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9e405640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d98161990 con 0x7f6d98069e90 2023-10-16T08:18:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d97fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d9815cd10 0x7f6d9815de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49276/0 (socket says 172.21.15.32:49276) 2023-10-16T08:18:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d97fff640 1 -- 172.21.15.32:0/4256891431 learned_addr learned my addr 172.21.15.32:0/4256891431 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9c9a2640 1 --2- 172.21.15.32:0/4256891431 >> v2:172.21.15.32:3300/0 conn(0x7f6d980ca5f0 0x7f6d980caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d977fe640 1 --2- 172.21.15.32:0/4256891431 >> v2:172.21.15.186:3300/0 conn(0x7f6d98069e90 0x7f6d9806a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d96ffd640 1 -- 172.21.15.32:0/4256891431 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622931477 0 0) 0x7f6d980ca150 con 0x7f6d9815cd10 2023-10-16T08:18:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9c9a2640 1 -- 172.21.15.32:0/4256891431 >> v2:172.21.15.186:3300/0 conn(0x7f6d98069e90 msgr2=0x7f6d9806a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9c9a2640 1 --2- 172.21.15.32:0/4256891431 >> v2:172.21.15.186:3300/0 conn(0x7f6d98069e90 0x7f6d9806a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9c9a2640 1 -- 172.21.15.32:0/4256891431 >> v1:172.21.15.32:6789/0 conn(0x7f6d9815cd10 legacy=0x7f6d9815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.257+0000 7f6d9c9a2640 1 -- 172.21.15.32:0/4256891431 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98160e10 con 0x7f6d980ca5f0 2023-10-16T08:18:36.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9c9a2640 1 --2- 172.21.15.32:0/4256891431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d980ca5f0 0x7f6d980caa40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6d980c6710 tx=0x7f6d9001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409719407d9a5559 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d96ffd640 1 -- 172.21.15.32:0/4256891431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900a59a0 con 0x7f6d980ca5f0 2023-10-16T08:18:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d96ffd640 1 -- 172.21.15.32:0/4256891431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d900a5b00 con 0x7f6d980ca5f0 2023-10-16T08:18:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d96ffd640 1 -- 172.21.15.32:0/4256891431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900a5e90 con 0x7f6d980ca5f0 2023-10-16T08:18:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- 172.21.15.32:0/4256891431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d980ca5f0 msgr2=0x7f6d980caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 --2- 172.21.15.32:0/4256891431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d980ca5f0 0x7f6d980caa40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6d980c6710 tx=0x7f6d9001c8e0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- 172.21.15.32:0/4256891431 shutdown_connections 2023-10-16T08:18:36.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 --2- 172.21.15.32:0/4256891431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d980ca5f0 0x7f6d980caa40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 --2- 172.21.15.32:0/4256891431 >> v2:172.21.15.186:3300/0 conn(0x7f6d98069e90 0x7f6d9806a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- 172.21.15.32:0/4256891431 >> 172.21.15.32:0/4256891431 conn(0x7f6d9806d310 msgr2=0x7f6d9806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:36.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- 172.21.15.32:0/4256891431 shutdown_connections 2023-10-16T08:18:36.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- 172.21.15.32:0/4256891431 wait complete. 2023-10-16T08:18:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 Processor -- start 2023-10-16T08:18:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- start start 2023-10-16T08:18:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 0x7f6d980c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d980ca5f0 0x7f6d980719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d9815cd10 0x7f6d981586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980c2b00 con 0x7f6d98069e90 2023-10-16T08:18:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980c41f0 con 0x7f6d9815cd10 2023-10-16T08:18:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.261+0000 7f6d9e405640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980c4440 con 0x7f6d980ca5f0 2023-10-16T08:18:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 0x7f6d980c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 0x7f6d980c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56820/0 (socket says 172.21.15.32:56820) 2023-10-16T08:18:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 -- 172.21.15.32:0/2582922881 learned_addr learned my addr 172.21.15.32:0/2582922881 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d9c9a2640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d9815cd10 0x7f6d981586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d977fe640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d980ca5f0 0x7f6d980719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 -- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d980ca5f0 msgr2=0x7f6d980719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d980ca5f0 0x7f6d980719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 -- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d9815cd10 msgr2=0x7f6d981586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d9815cd10 0x7f6d981586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 -- 172.21.15.32:0/2582922881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90009a70 con 0x7f6d98069e90 2023-10-16T08:18:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d97fff640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 0x7f6d980c25c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c480 tx=0x7f6d8800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880074e0 con 0x7f6d98069e90 2023-10-16T08:18:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88007660 con 0x7f6d98069e90 2023-10-16T08:18:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88004a30 con 0x7f6d98069e90 2023-10-16T08:18:36.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980c4720 con 0x7f6d98069e90 2023-10-16T08:18:36.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980c4ce0 con 0x7f6d98069e90 2023-10-16T08:18:36.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d98074010 con 0x7f6d98069e90 2023-10-16T08:18:36.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.265+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6d88004b90 con 0x7f6d98069e90 2023-10-16T08:18:36.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.269+0000 7f6d94ff9640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d840664d0 0x7f6d84068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.269+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f6d880832d0 con 0x7f6d98069e90 2023-10-16T08:18:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.269+0000 7f6d977fe640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d840664d0 0x7f6d84068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.269+0000 7f6d977fe640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d840664d0 0x7f6d84068990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d980c6520 tx=0x7f6d8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.269+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6d88053ee0 con 0x7f6d98069e90 2023-10-16T08:18:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.461+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d980b1a60 con 0x7f6d98069e90 2023-10-16T08:18:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d94ff9640 1 -- 172.21.15.32:0/2582922881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f6d88053a70 con 0x7f6d98069e90 2023-10-16T08:18:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:36.470 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:17:51.541347+0000","modified":"2023-10-16T08:18:01.158878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/1556343693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1556343693},{"type":"v1","addr":"172.21.15.32:6835","nonce":1556343693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T08:18:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2023-10-16T08:18:36.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d840664d0 msgr2=0x7f6d84068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d840664d0 0x7f6d84068990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d980c6520 tx=0x7f6d8c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:36.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 msgr2=0x7f6d980c25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 0x7f6d980c25c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c480 tx=0x7f6d8800c940 comp rx=0 tx=0).stop 2023-10-16T08:18:36.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 shutdown_connections 2023-10-16T08:18:36.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d9815cd10 0x7f6d981586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f6d840664d0 0x7f6d84068990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f6d980ca5f0 0x7f6d980719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 --2- 172.21.15.32:0/2582922881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d98069e90 0x7f6d980c25c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 >> 172.21.15.32:0/2582922881 conn(0x7f6d9806d310 msgr2=0x7f6d980b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 shutdown_connections 2023-10-16T08:18:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.465+0000 7f6d9e405640 1 -- 172.21.15.32:0/2582922881 wait complete. 2023-10-16T08:18:36.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:36.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddca2f7640 1 Processor -- start 2023-10-16T08:18:36.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddca2f7640 1 -- start start 2023-10-16T08:18:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddca2f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc40d3010 0x7fddc40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddca2f7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40ce770 con 0x7fddc40d15d0 2023-10-16T08:18:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddca2f7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40d0600 con 0x7fddc415cb90 2023-10-16T08:18:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddca2f7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40c6450 con 0x7fddc40d3010 2023-10-16T08:18:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fddc40d15d0 0x7fddc40c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49288/0 (socket says 172.21.15.32:49288) 2023-10-16T08:18:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc37fe640 1 -- 172.21.15.32:0/3042328263 learned_addr learned my addr 172.21.15.32:0/3042328263 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc8894640 1 --2- 172.21.15.32:0/3042328263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc40d3010 0x7fddc40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027645619 0 0) 0x7fddc40ce770 con 0x7fddc40d15d0 2023-10-16T08:18:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac003530 con 0x7fddc40d15d0 2023-10-16T08:18:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9570379 0 0) 0x7fddc40d0600 con 0x7fddc415cb90 2023-10-16T08:18:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc40ce770 con 0x7fddc415cb90 2023-10-16T08:18:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473784372 0 0) 0x7fddac003530 con 0x7fddc40d15d0 2023-10-16T08:18:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc8894640 1 -- 172.21.15.32:0/3042328263 >> v1:172.21.15.186:6789/0 conn(0x7fddc415cb90 legacy=0x7fddc40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc8894640 1 -- 172.21.15.32:0/3042328263 >> v1:172.21.15.32:6789/0 conn(0x7fddc40d15d0 legacy=0x7fddc40c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.629+0000 7fddc8894640 1 -- 172.21.15.32:0/3042328263 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40d08a0 con 0x7fddc40d3010 2023-10-16T08:18:36.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddc8894640 1 --2- 172.21.15.32:0/3042328263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc40d3010 0x7fddc40cfb10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009dc0 tx=0x7fddbc01c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e53fd8edf3eca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8001680 con 0x7fddc40d3010 2023-10-16T08:18:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc0a5610 con 0x7fddc40d3010 2023-10-16T08:18:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddc2ffd640 1 -- 172.21.15.32:0/3042328263 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0a59a0 con 0x7fddc40d3010 2023-10-16T08:18:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 -- 172.21.15.32:0/3042328263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc40d3010 msgr2=0x7fddc40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 --2- 172.21.15.32:0/3042328263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc40d3010 0x7fddc40cfb10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009dc0 tx=0x7fddbc01c7d0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 -- 172.21.15.32:0/3042328263 shutdown_connections 2023-10-16T08:18:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 --2- 172.21.15.32:0/3042328263 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc40d3010 0x7fddc40cfb10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 -- 172.21.15.32:0/3042328263 >> 172.21.15.32:0/3042328263 conn(0x7fddc409bc50 msgr2=0x7fddc40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 -- 172.21.15.32:0/3042328263 shutdown_connections 2023-10-16T08:18:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.633+0000 7fddca2f7640 1 -- 172.21.15.32:0/3042328263 wait complete. 2023-10-16T08:18:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 Processor -- start 2023-10-16T08:18:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 -- start start 2023-10-16T08:18:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc40d15d0 0x7fddc41f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 0x7fddc406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc415cb90 0x7fddc406b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4159840 con 0x7fddc40d3010 2023-10-16T08:18:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40c5fc0 con 0x7fddc40d15d0 2023-10-16T08:18:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4158bc0 con 0x7fddc415cb90 2023-10-16T08:18:36.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 0x7fddc406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 0x7fddc406b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56824/0 (socket says 172.21.15.32:56824) 2023-10-16T08:18:36.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 -- 172.21.15.32:0/2121638337 learned_addr learned my addr 172.21.15.32:0/2121638337 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:36.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 -- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc415cb90 msgr2=0x7fddc406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc8894640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc415cb90 0x7fddc406b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc3fff640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc40d15d0 0x7fddc41f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc415cb90 0x7fddc406b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 -- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc40d15d0 msgr2=0x7fddc41f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc40d15d0 0x7fddc41f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 -- 172.21.15.32:0/2121638337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc009a70 con 0x7fddc40d3010 2023-10-16T08:18:36.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc8894640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc415cb90 0x7fddc406b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:36.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc3fff640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc40d15d0 0x7fddc41f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:36.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc37fe640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 0x7fddc406b3d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c780 tx=0x7fddb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005cf0 con 0x7fddc40d3010 2023-10-16T08:18:36.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8005e70 con 0x7fddc40d3010 2023-10-16T08:18:36.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007290 con 0x7fddc40d3010 2023-10-16T08:18:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4158ea0 con 0x7fddc40d3010 2023-10-16T08:18:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.637+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc4159460 con 0x7fddc40d3010 2023-10-16T08:18:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.641+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc40d38a0 con 0x7fddc40d3010 2023-10-16T08:18:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.641+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fddb80073f0 con 0x7fddc40d3010 2023-10-16T08:18:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.641+0000 7fddc0ff9640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fddb00664d0 0x7fddb0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.641+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fddb8083fa0 con 0x7fddc40d3010 2023-10-16T08:18:36.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.641+0000 7fddc3fff640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fddb00664d0 0x7fddb0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.641+0000 7fddc3fff640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fddb00664d0 0x7fddb0068990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fddb4001240 tx=0x7fddb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.645+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fddb8051090 con 0x7fddc40d3010 2023-10-16T08:18:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.801+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddc4159980 con 0x7fddc40d3010 2023-10-16T08:18:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddc0ff9640 1 -- 172.21.15.32:0/2121638337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18578 (secure 0 0 0) 0x7fddc4159980 con 0x7fddc40d3010 2023-10-16T08:18:36.809 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:36.810 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:02.739979+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:17:49.375624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:17:50.533974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-16T08:18:00.037246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fddb00664d0 msgr2=0x7fddb0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fddb00664d0 0x7fddb0068990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fddb4001240 tx=0x7fddb4009040 comp rx=0 tx=0).stop 2023-10-16T08:18:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 msgr2=0x7fddc406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 0x7fddc406b3d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c780 tx=0x7fddb800cc40 comp rx=0 tx=0).stop 2023-10-16T08:18:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 shutdown_connections 2023-10-16T08:18:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fddb00664d0 0x7fddb0068990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fddc415cb90 0x7fddc406b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc40d3010 0x7fddc406b3d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 --2- 172.21.15.32:0/2121638337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddc40d15d0 0x7fddc41f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 >> 172.21.15.32:0/2121638337 conn(0x7fddc409bc50 msgr2=0x7fddc40d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 shutdown_connections 2023-10-16T08:18:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.805+0000 7fddca2f7640 1 -- 172.21.15.32:0/2121638337 wait complete. 2023-10-16T08:18:36.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail testcephfs1 2023-10-16T08:18:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.969+0000 7f765773e640 1 Processor -- start 2023-10-16T08:18:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.969+0000 7f765773e640 1 -- start start 2023-10-16T08:18:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f765773e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f765015cb90 0x7f76500cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f765773e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f76500d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f765773e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500ce770 con 0x7f76500d1400 2023-10-16T08:18:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f765773e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500d1890 con 0x7f765015cb90 2023-10-16T08:18:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f765773e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500d0600 con 0x7f76500d3010 2023-10-16T08:18:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7655cdb640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76500d1400 0x7f76500cd3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49300/0 (socket says 172.21.15.32:49300) 2023-10-16T08:18:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7655cdb640 1 -- 172.21.15.32:0/687118676 learned_addr learned my addr 172.21.15.32:0/687118676 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7654cd9640 1 --2- 172.21.15.32:0/687118676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f76500d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f76554da640 1 --2- 172.21.15.32:0/687118676 >> v2:172.21.15.186:3300/0 conn(0x7f765015cb90 0x7f76500cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7647fff640 1 -- 172.21.15.32:0/687118676 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010171917 0 0) 0x7f76500ce770 con 0x7f76500d1400 2023-10-16T08:18:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7647fff640 1 -- 172.21.15.32:0/687118676 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f763c003530 con 0x7f76500d1400 2023-10-16T08:18:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7647fff640 1 -- 172.21.15.32:0/687118676 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981579330 0 0) 0x7f763c003530 con 0x7f76500d1400 2023-10-16T08:18:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7654cd9640 1 -- 172.21.15.32:0/687118676 >> v2:172.21.15.186:3300/0 conn(0x7f765015cb90 msgr2=0x7f76500cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7654cd9640 1 --2- 172.21.15.32:0/687118676 >> v2:172.21.15.186:3300/0 conn(0x7f765015cb90 0x7f76500cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7654cd9640 1 -- 172.21.15.32:0/687118676 >> v1:172.21.15.32:6789/0 conn(0x7f76500d1400 legacy=0x7f76500cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.973+0000 7f7654cd9640 1 -- 172.21.15.32:0/687118676 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500c6450 con 0x7f76500d3010 2023-10-16T08:18:36.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f7654cd9640 1 --2- 172.21.15.32:0/687118676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f76500d0ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76400097c0 tx=0x7f764001c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21161e27e1d76bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f7647fff640 1 -- 172.21.15.32:0/687118676 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648001a00 con 0x7f76500d3010 2023-10-16T08:18:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f7647fff640 1 -- 172.21.15.32:0/687118676 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76400a7050 con 0x7f76500d3010 2023-10-16T08:18:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f7647fff640 1 -- 172.21.15.32:0/687118676 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400ab620 con 0x7f76500d3010 2023-10-16T08:18:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 -- 172.21.15.32:0/687118676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 msgr2=0x7f76500d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 --2- 172.21.15.32:0/687118676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f76500d0ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76400097c0 tx=0x7f764001c2e0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 -- 172.21.15.32:0/687118676 shutdown_connections 2023-10-16T08:18:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 --2- 172.21.15.32:0/687118676 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f76500d0ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 --2- 172.21.15.32:0/687118676 >> v2:172.21.15.186:3300/0 conn(0x7f765015cb90 0x7f76500cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 -- 172.21.15.32:0/687118676 >> 172.21.15.32:0/687118676 conn(0x7f765009bc50 msgr2=0x7f76500d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 -- 172.21.15.32:0/687118676 shutdown_connections 2023-10-16T08:18:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 -- 172.21.15.32:0/687118676 wait complete. 2023-10-16T08:18:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 Processor -- start 2023-10-16T08:18:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.977+0000 7f765773e640 1 -- start start 2023-10-16T08:18:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f765773e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 0x7f7650156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f765773e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f7650153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f765773e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765015cb90 0x7f7650159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f765773e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500b4b40 con 0x7f76500d1400 2023-10-16T08:18:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f765773e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650154120 con 0x7f765015cb90 2023-10-16T08:18:36.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f765773e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7650155b80 con 0x7f76500d3010 2023-10-16T08:18:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 0x7f7650156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 0x7f7650156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56826/0 (socket says 172.21.15.32:56826) 2023-10-16T08:18:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 -- 172.21.15.32:0/2167136188 learned_addr learned my addr 172.21.15.32:0/2167136188 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f7654cd9640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f7650153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f7655cdb640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765015cb90 0x7f7650159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 -- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 msgr2=0x7f7650153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f7650153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 -- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765015cb90 msgr2=0x7f7650159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765015cb90 0x7f7650159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 -- 172.21.15.32:0/2167136188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640009420 con 0x7f76500d1400 2023-10-16T08:18:36.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f76554da640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 0x7f7650156d50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e9b0 tx=0x7f764c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.981+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c009670 con 0x7f76500d1400 2023-10-16T08:18:36.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00c9a0 con 0x7f76500d1400 2023-10-16T08:18:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00cd10 con 0x7f76500d1400 2023-10-16T08:18:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76500cac90 con 0x7f76500d1400 2023-10-16T08:18:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7650154340 con 0x7f76500d1400 2023-10-16T08:18:36.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7650074010 con 0x7f76500d1400 2023-10-16T08:18:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f764c013470 con 0x7f76500d1400 2023-10-16T08:18:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f7645ffb640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.985+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f764c0847e0 con 0x7f76500d1400 2023-10-16T08:18:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.989+0000 7f7654cd9640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:36.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.989+0000 7f7654cd9640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f76400ad000 tx=0x7f7640008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:36.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:36.993+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f764c0518d0 con 0x7f76500d1400 2023-10-16T08:18:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.181+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "testcephfs1"} v 0) v1 -- 0x7f76500cfb40 con 0x7f76500d1400 2023-10-16T08:18:37.188 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-10-16T08:18:37.181+0000 7efeb4e3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T08:18:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.185+0000 7f7645ffb640 1 -- 172.21.15.32:0/2167136188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "testcephfs1"}]=0 testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f764c054ef0 con 0x7f76500d1400 2023-10-16T08:18:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T08:18:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 msgr2=0x7f7634068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f76400ad000 tx=0x7f7640008040 comp rx=0 tx=0).stop 2023-10-16T08:18:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 msgr2=0x7f7650156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 0x7f7650156d50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e9b0 tx=0x7f764c00ee70 comp rx=0 tx=0).stop 2023-10-16T08:18:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 shutdown_connections 2023-10-16T08:18:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f765015cb90 0x7f7650159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f76340664a0 0x7f7634068960 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f76500d3010 0x7f7650153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 --2- 172.21.15.32:0/2167136188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76500d1400 0x7f7650156d50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 >> 172.21.15.32:0/2167136188 conn(0x7f765009bc50 msgr2=0x7f765006b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 shutdown_connections 2023-10-16T08:18:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.189+0000 7f765773e640 1 -- 172.21.15.32:0/2167136188 wait complete. 2023-10-16T08:18:37.202 INFO:tasks.ceph.mds.c.smithi032.stderr: -3574> 2023-10-16T08:17:56.121+0000 7f90095ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:18:37.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm testcephfs1 --yes-i-really-mean-it 2023-10-16T08:18:37.251 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-10-16T08:18:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 Processor -- start 2023-10-16T08:18:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 -- start start 2023-10-16T08:18:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f65f815cc60 0x7f65f80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65f80ceda0 0x7f65f80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f80cc330 con 0x7f65f8161260 2023-10-16T08:18:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f80cb720 con 0x7f65f80ceda0 2023-10-16T08:18:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65fc846640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f80cba00 con 0x7f65f815cc60 2023-10-16T08:18:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65f80ceda0 0x7f65f80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f5d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65f8161260 0x7f65f80ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49304/0 (socket says 172.21.15.32:49304) 2023-10-16T08:18:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f5d9b640 1 -- 172.21.15.32:0/1832834136 learned_addr learned my addr 172.21.15.32:0/1832834136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f659c640 1 --2- 172.21.15.32:0/1832834136 >> v2:172.21.15.186:3300/0 conn(0x7f65f815cc60 0x7f65f80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f6d9d640 1 -- 172.21.15.32:0/1832834136 >> v2:172.21.15.186:3300/0 conn(0x7f65f815cc60 msgr2=0x7f65f80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f6d9d640 1 --2- 172.21.15.32:0/1832834136 >> v2:172.21.15.186:3300/0 conn(0x7f65f815cc60 0x7f65f80cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f6d9d640 1 -- 172.21.15.32:0/1832834136 >> v1:172.21.15.32:6789/0 conn(0x7f65f8161260 legacy=0x7f65f80ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f559a640 1 -- 172.21.15.32:0/1832834136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821161596 0 0) 0x7f65f80cc330 con 0x7f65f8161260 2023-10-16T08:18:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.357+0000 7f65f6d9d640 1 -- 172.21.15.32:0/1832834136 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80cbce0 con 0x7f65f80ceda0 2023-10-16T08:18:37.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65f6d9d640 1 --2- 172.21.15.32:0/1832834136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f80ceda0 0x7f65f80cc990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f65f80d0810 tx=0x7f65f001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31393dba854c46a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65f559a640 1 -- 172.21.15.32:0/1832834136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00a59c0 con 0x7f65f80ceda0 2023-10-16T08:18:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65f559a640 1 -- 172.21.15.32:0/1832834136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f00a5b20 con 0x7f65f80ceda0 2023-10-16T08:18:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65f559a640 1 -- 172.21.15.32:0/1832834136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00ad440 con 0x7f65f80ceda0 2023-10-16T08:18:37.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 -- 172.21.15.32:0/1832834136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f80ceda0 msgr2=0x7f65f80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 --2- 172.21.15.32:0/1832834136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f80ceda0 0x7f65f80cc990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f65f80d0810 tx=0x7f65f001c8e0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 -- 172.21.15.32:0/1832834136 shutdown_connections 2023-10-16T08:18:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 --2- 172.21.15.32:0/1832834136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f80ceda0 0x7f65f80cc990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 --2- 172.21.15.32:0/1832834136 >> v2:172.21.15.186:3300/0 conn(0x7f65f815cc60 0x7f65f80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 -- 172.21.15.32:0/1832834136 >> 172.21.15.32:0/1832834136 conn(0x7f65f806e570 msgr2=0x7f65f806e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:37.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 -- 172.21.15.32:0/1832834136 shutdown_connections 2023-10-16T08:18:37.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.361+0000 7f65fc846640 1 -- 172.21.15.32:0/1832834136 wait complete. 2023-10-16T08:18:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 Processor -- start 2023-10-16T08:18:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 -- start start 2023-10-16T08:18:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f80ceda0 0x7f65f80c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:37.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f815cc60 0x7f65f815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 0x7f65f815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c5620 con 0x7f65f8161260 2023-10-16T08:18:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c4a10 con 0x7f65f815cc60 2023-10-16T08:18:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c4cf0 con 0x7f65f80ceda0 2023-10-16T08:18:37.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 0x7f65f815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 0x7f65f815fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56862/0 (socket says 172.21.15.32:56862) 2023-10-16T08:18:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 -- 172.21.15.32:0/3030100376 learned_addr learned my addr 172.21.15.32:0/3030100376 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f659c640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f80ceda0 0x7f65f80c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f5d9b640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f815cc60 0x7f65f815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 -- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f80ceda0 msgr2=0x7f65f80c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f80ceda0 0x7f65f80c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 -- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f815cc60 msgr2=0x7f65f815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f815cc60 0x7f65f815f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 -- 172.21.15.32:0/3030100376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f0009a70 con 0x7f65f8161260 2023-10-16T08:18:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f5d9b640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f815cc60 0x7f65f815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f659c640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f80ceda0 0x7f65f80c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65f6d9d640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 0x7f65f815fa50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f65f001cdc0 tx=0x7f65f0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00bf070 con 0x7f65f8161260 2023-10-16T08:18:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f00aec10 con 0x7f65f8161260 2023-10-16T08:18:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f80c4fd0 con 0x7f65f8161260 2023-10-16T08:18:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.365+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00b7410 con 0x7f65f8161260 2023-10-16T08:18:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f80c63e0 con 0x7f65f8161260 2023-10-16T08:18:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f8161690 con 0x7f65f8161260 2023-10-16T08:18:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f65f00b4050 con 0x7f65f8161260 2023-10-16T08:18:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65e6ffd640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f65d80664a0 0x7f65d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f65f011ef60 con 0x7f65f8161260 2023-10-16T08:18:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65f659c640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f65d80664a0 0x7f65d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.369+0000 7f65f659c640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f65d80664a0 0x7f65d8068960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009840 tx=0x7f65e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.373+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f65f8161690 con 0x7f65f8161260 2023-10-16T08:18:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:37.565+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f65f8161690 con 0x7f65f8161260 2023-10-16T08:18:38.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65e6ffd640 1 -- 172.21.15.32:0/3030100376 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 109+0+0 (secure 0 0 0) 0x7f65f8161690 con 0x7f65f8161260 2023-10-16T08:18:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f65d80664a0 msgr2=0x7f65d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f65d80664a0 0x7f65d8068960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009840 tx=0x7f65e8008040 comp rx=0 tx=0).stop 2023-10-16T08:18:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 msgr2=0x7f65f815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:38.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 0x7f65f815fa50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f65f001cdc0 tx=0x7f65f0019680 comp rx=0 tx=0).stop 2023-10-16T08:18:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 shutdown_connections 2023-10-16T08:18:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65f8161260 0x7f65f815fa50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65f815cc60 0x7f65f815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f65d80664a0 0x7f65d8068960 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 --2- 172.21.15.32:0/3030100376 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f65f80ceda0 0x7f65f80c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 >> 172.21.15.32:0/3030100376 conn(0x7f65f806e570 msgr2=0x7f65f806b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 shutdown_connections 2023-10-16T08:18:38.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.189+0000 7f65fc846640 1 -- 172.21.15.32:0/3030100376 wait complete. 2023-10-16T08:18:38.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_metadata testcephfs1_metadata --yes-i-really-really-mean-it 2023-10-16T08:18:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 Processor -- start 2023-10-16T08:18:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 -- start start 2023-10-16T08:18:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b0159980 0x7fe1b00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1b006cbc0 0x7fe1b00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:38.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b00c8a20 con 0x7fe1b01595a0 2023-10-16T08:18:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b015e300 con 0x7fe1b006cbc0 2023-10-16T08:18:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b6b65640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00c8c10 con 0x7fe1b0159980 2023-10-16T08:18:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1b006cbc0 0x7fe1b00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1b006cbc0 0x7fe1b00b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56864/0 (socket says 172.21.15.32:56864) 2023-10-16T08:18:38.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 -- 172.21.15.32:0/3799158288 learned_addr learned my addr 172.21.15.32:0/3799158288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1affff640 1 --2- 172.21.15.32:0/3799158288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b0159980 0x7fe1b00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 -- 172.21.15.32:0/3799158288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b0159980 msgr2=0x7fe1b00b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 --2- 172.21.15.32:0/3799158288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b0159980 0x7fe1b00b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 -- 172.21.15.32:0/3799158288 >> v1:172.21.15.186:6789/0 conn(0x7fe1b01595a0 legacy=0x7fe1b006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.405+0000 7fe1b5102640 1 -- 172.21.15.32:0/3799158288 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00c8ef0 con 0x7fe1b006cbc0 2023-10-16T08:18:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b5102640 1 --2- 172.21.15.32:0/3799158288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b006cbc0 0x7fe1b00b4100 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009e50 tx=0x7fe1a801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb2f0b010e8708 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1af7fe640 1 -- 172.21.15.32:0/3799158288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80a56b0 con 0x7fe1b006cbc0 2023-10-16T08:18:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1af7fe640 1 -- 172.21.15.32:0/3799158288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80a5810 con 0x7fe1b006cbc0 2023-10-16T08:18:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1af7fe640 1 -- 172.21.15.32:0/3799158288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80a5ba0 con 0x7fe1b006cbc0 2023-10-16T08:18:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 -- 172.21.15.32:0/3799158288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b006cbc0 msgr2=0x7fe1b00b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/3799158288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b006cbc0 0x7fe1b00b4100 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009e50 tx=0x7fe1a801c940 comp rx=0 tx=0).stop 2023-10-16T08:18:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 -- 172.21.15.32:0/3799158288 shutdown_connections 2023-10-16T08:18:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/3799158288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b006cbc0 0x7fe1b00b4100 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/3799158288 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b0159980 0x7fe1b00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 -- 172.21.15.32:0/3799158288 >> 172.21.15.32:0/3799158288 conn(0x7fe1b006a820 msgr2=0x7fe1b00b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 -- 172.21.15.32:0/3799158288 shutdown_connections 2023-10-16T08:18:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 -- 172.21.15.32:0/3799158288 wait complete. 2023-10-16T08:18:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 Processor -- start 2023-10-16T08:18:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.409+0000 7fe1b6b65640 1 -- start start 2023-10-16T08:18:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b006cbc0 0x7fe1b0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b01595a0 0x7fe1b0157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 0x7fe1b0071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0071dd0 con 0x7fe1b00713e0 2023-10-16T08:18:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b015ab10 con 0x7fe1b006cbc0 2023-10-16T08:18:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b015be20 con 0x7fe1b01595a0 2023-10-16T08:18:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 0x7fe1b0071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 0x7fe1b0071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56874/0 (socket says 172.21.15.32:56874) 2023-10-16T08:18:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 -- 172.21.15.32:0/1846697874 learned_addr learned my addr 172.21.15.32:0/1846697874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1affff640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b01595a0 0x7fe1b0157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b4901640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b006cbc0 0x7fe1b0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 -- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b01595a0 msgr2=0x7fe1b0157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b01595a0 0x7fe1b0157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 -- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b006cbc0 msgr2=0x7fe1b0150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b006cbc0 0x7fe1b0150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:38.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 -- 172.21.15.32:0/1846697874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a8009b30 con 0x7fe1b00713e0 2023-10-16T08:18:38.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b5102640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 0x7fe1b0071890 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009e50 tx=0x7fe1a80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:38.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80b8670 con 0x7fe1b00713e0 2023-10-16T08:18:38.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80b87d0 con 0x7fe1b00713e0 2023-10-16T08:18:38.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80b8b60 con 0x7fe1b00713e0 2023-10-16T08:18:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b015ad80 con 0x7fe1b00713e0 2023-10-16T08:18:38.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b015b340 con 0x7fe1b00713e0 2023-10-16T08:18:38.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.413+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b004e170 con 0x7fe1b00713e0 2023-10-16T08:18:38.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.417+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe1a80b8cc0 con 0x7fe1b00713e0 2023-10-16T08:18:38.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.417+0000 7fe1ad7fa640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1940664a0 0x7fe194068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:38.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.417+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6535+0+0 (secure 0 0 0) 0x7fe1a811f5c0 con 0x7fe1b00713e0 2023-10-16T08:18:38.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.417+0000 7fe1b4901640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1940664a0 0x7fe194068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:38.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.417+0000 7fe1b4901640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1940664a0 0x7fe194068960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4007fd0 tx=0x7fe1a4007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:38.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.421+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe1a80ec8c0 con 0x7fe1b00713e0 2023-10-16T08:18:38.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:38.577+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe1b00b4090 con 0x7fe1b00713e0 2023-10-16T08:18:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.189+0000 7fe1ad7fa640 1 -- 172.21.15.32:0/1846697874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_metadata' removed v44) v1 ==== 196+0+0 (secure 0 0 0) 0x7fe1a80efee0 con 0x7fe1b00713e0 2023-10-16T08:18:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_metadata' removed 2023-10-16T08:18:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.189+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1940664a0 msgr2=0x7fe194068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.189+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1940664a0 0x7fe194068960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4007fd0 tx=0x7fe1a4007e40 comp rx=0 tx=0).stop 2023-10-16T08:18:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.189+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 msgr2=0x7fe1b0071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.189+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 0x7fe1b0071890 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009e50 tx=0x7fe1a80196e0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 shutdown_connections 2023-10-16T08:18:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b00713e0 0x7fe1b0071890 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1940664a0 0x7fe194068960 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe1b01595a0 0x7fe1b0157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 --2- 172.21.15.32:0/1846697874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b006cbc0 0x7fe1b0150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 >> 172.21.15.32:0/1846697874 conn(0x7fe1b006a820 msgr2=0x7fe1b00c78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 shutdown_connections 2023-10-16T08:18:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.193+0000 7fe1b6b65640 1 -- 172.21.15.32:0/1846697874 wait complete. 2023-10-16T08:18:39.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_data testcephfs1_data --yes-i-really-really-mean-it 2023-10-16T08:18:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.369+0000 7f5a23310640 1 Processor -- start 2023-10-16T08:18:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.369+0000 7f5a23310640 1 -- start start 2023-10-16T08:18:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a23310640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a23310640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a1c15de90 0x7f5a1c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a23310640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c070ce0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a23310640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c15f230 con 0x7f5a1c15de90 2023-10-16T08:18:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a23310640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0700d0 con 0x7f5a1c15dab0 2023-10-16T08:18:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a1c15de90 0x7f5a1c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a1c15de90 0x7f5a1c0cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56878/0 (socket says 172.21.15.32:56878) 2023-10-16T08:18:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 -- 172.21.15.32:0/774666611 learned_addr learned my addr 172.21.15.32:0/774666611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:39.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a2230e640 1 --2- 172.21.15.32:0/774666611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 -- 172.21.15.32:0/774666611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 msgr2=0x7f5a1c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 --2- 172.21.15.32:0/774666611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 -- 172.21.15.32:0/774666611 >> v1:172.21.15.186:6789/0 conn(0x7f5a1c0b3e50 legacy=0x7f5a1c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:39.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.373+0000 7f5a21b0d640 1 -- 172.21.15.32:0/774666611 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0703b0 con 0x7f5a1c15de90 2023-10-16T08:18:39.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a21b0d640 1 --2- 172.21.15.32:0/774666611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0cab90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0098a0 tx=0x7f5a0c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b7bba5a224a47d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a2130c640 1 -- 172.21.15.32:0/774666611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0ac630 con 0x7f5a1c15de90 2023-10-16T08:18:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a2130c640 1 -- 172.21.15.32:0/774666611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0ac790 con 0x7f5a1c15de90 2023-10-16T08:18:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a2130c640 1 -- 172.21.15.32:0/774666611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0acb20 con 0x7f5a1c15de90 2023-10-16T08:18:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 -- 172.21.15.32:0/774666611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c15de90 msgr2=0x7f5a1c0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 --2- 172.21.15.32:0/774666611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0cab90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0098a0 tx=0x7f5a0c01c3d0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 -- 172.21.15.32:0/774666611 shutdown_connections 2023-10-16T08:18:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 --2- 172.21.15.32:0/774666611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0cab90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 --2- 172.21.15.32:0/774666611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 -- 172.21.15.32:0/774666611 >> 172.21.15.32:0/774666611 conn(0x7f5a1c06a6e0 msgr2=0x7f5a1c0afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 -- 172.21.15.32:0/774666611 shutdown_connections 2023-10-16T08:18:39.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 -- 172.21.15.32:0/774666611 wait complete. 2023-10-16T08:18:39.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 Processor -- start 2023-10-16T08:18:39.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.377+0000 7f5a23310640 1 -- start start 2023-10-16T08:18:39.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 0x7f5a1c157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:39.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c15a6b0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c1f97c0 con 0x7f5a1c15de90 2023-10-16T08:18:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c1583b0 con 0x7f5a1c15dab0 2023-10-16T08:18:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 0x7f5a1c157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 0x7f5a1c157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56894/0 (socket says 172.21.15.32:56894) 2023-10-16T08:18:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 -- 172.21.15.32:0/950871734 learned_addr learned my addr 172.21.15.32:0/950871734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a21b0d640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a22b0f640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 -- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 msgr2=0x7f5a1c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 -- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a1c15de90 msgr2=0x7f5a1c0aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 -- 172.21.15.32:0/950871734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0094d0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a21b0d640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a2230e640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 0x7f5a1c157e70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800d830 tx=0x7f5a1800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800b6c0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1800b820 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800bb90 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c159750 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c159d10 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.381+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c04e9c0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.385+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5a1801e460 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.385+0000 7f5a12ffd640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5a000664a0 0x7f5a00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:39.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.385+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f5a180852c0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.385+0000 7f5a21b0d640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5a000664a0 0x7f5a00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.385+0000 7f5a21b0d640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5a000664a0 0x7f5a00068960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0ae000 tx=0x7f5a0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.385+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5a180525a0 con 0x7f5a1c0b3e50 2023-10-16T08:18:39.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:39.545+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a1c0b0b90 con 0x7f5a1c0b3e50 2023-10-16T08:18:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.189+0000 7f5a12ffd640 1 -- 172.21.15.32:0/950871734 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_data' removed v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5a18055bc0 con 0x7f5a1c0b3e50 2023-10-16T08:18:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_data' removed 2023-10-16T08:18:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5a000664a0 msgr2=0x7f5a00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5a000664a0 0x7f5a00068960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0ae000 tx=0x7f5a0c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 msgr2=0x7f5a1c157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 0x7f5a1c157e70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800d830 tx=0x7f5a1800dcf0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 shutdown_connections 2023-10-16T08:18:40.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5a1c15de90 0x7f5a1c0aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5a000664a0 0x7f5a00068960 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5a1c15dab0 0x7f5a1c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 --2- 172.21.15.32:0/950871734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0b3e50 0x7f5a1c157e70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 >> 172.21.15.32:0/950871734 conn(0x7f5a1c06a6e0 msgr2=0x7f5a1c15c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:40.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 shutdown_connections 2023-10-16T08:18:40.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.193+0000 7f5a23310640 1 -- 172.21.15.32:0/950871734 wait complete. 2023-10-16T08:18:40.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-10-16T08:18:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.369+0000 7fc1a2bc4640 1 Processor -- start 2023-10-16T08:18:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a2bc4640 1 -- start start 2023-10-16T08:18:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a2bc4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc19c153540 0x7fc19c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a2bc4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc19c0cb6e0 0x7fc19c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:40.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a2bc4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c159510 con 0x7fc19c0afaa0 2023-10-16T08:18:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a2bc4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c159790 con 0x7fc19c0cb6e0 2023-10-16T08:18:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a2bc4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c159a70 con 0x7fc19c153540 2023-10-16T08:18:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc19c0cb6e0 0x7fc19c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc19c0cb6e0 0x7fc19c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56910/0 (socket says 172.21.15.32:56910) 2023-10-16T08:18:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 -- 172.21.15.32:0/3839142871 learned_addr learned my addr 172.21.15.32:0/3839142871 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc19bfff640 1 --1- 172.21.15.32:0/3839142871 >> v1:172.21.15.32:6789/0 conn(0x7fc19c0afaa0 0x7fc19c0cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49314/0 (socket says 172.21.15.32:49314) 2023-10-16T08:18:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a0960640 1 --2- 172.21.15.32:0/3839142871 >> v2:172.21.15.186:3300/0 conn(0x7fc19c153540 0x7fc19c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 -- 172.21.15.32:0/3839142871 >> v2:172.21.15.186:3300/0 conn(0x7fc19c153540 msgr2=0x7fc19c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 --2- 172.21.15.32:0/3839142871 >> v2:172.21.15.186:3300/0 conn(0x7fc19c153540 0x7fc19c0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 -- 172.21.15.32:0/3839142871 >> v1:172.21.15.32:6789/0 conn(0x7fc19c0afaa0 legacy=0x7fc19c0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:40.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.373+0000 7fc1a1161640 1 -- 172.21.15.32:0/3839142871 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c159d50 con 0x7fc19c0cb6e0 2023-10-16T08:18:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a1161640 1 --2- 172.21.15.32:0/3839142871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0cb6e0 0x7fc19c15ccb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc19c06e7b0 tx=0x7fc19401eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d0335606768b01 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc19b7fe640 1 -- 172.21.15.32:0/3839142871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940a7b60 con 0x7fc19c0cb6e0 2023-10-16T08:18:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc19b7fe640 1 -- 172.21.15.32:0/3839142871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940ab070 con 0x7fc19c0cb6e0 2023-10-16T08:18:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc19b7fe640 1 -- 172.21.15.32:0/3839142871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940b1400 con 0x7fc19c0cb6e0 2023-10-16T08:18:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3839142871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0cb6e0 msgr2=0x7fc19c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3839142871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0cb6e0 0x7fc19c15ccb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc19c06e7b0 tx=0x7fc19401eb10 comp rx=0 tx=0).stop 2023-10-16T08:18:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3839142871 shutdown_connections 2023-10-16T08:18:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3839142871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0cb6e0 0x7fc19c15ccb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3839142871 >> v2:172.21.15.186:3300/0 conn(0x7fc19c153540 0x7fc19c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3839142871 >> 172.21.15.32:0/3839142871 conn(0x7fc19c06a820 msgr2=0x7fc19c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3839142871 shutdown_connections 2023-10-16T08:18:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3839142871 wait complete. 2023-10-16T08:18:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 Processor -- start 2023-10-16T08:18:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.377+0000 7fc1a2bc4640 1 -- start start 2023-10-16T08:18:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 0x7fc19c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c0cb6e0 0x7fc19c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c153540 0x7fc19c13afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0b4240 con 0x7fc19c0afaa0 2023-10-16T08:18:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c1473d0 con 0x7fc19c153540 2023-10-16T08:18:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0720f0 con 0x7fc19c0cb6e0 2023-10-16T08:18:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 0x7fc19c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 0x7fc19c0b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56924/0 (socket says 172.21.15.32:56924) 2023-10-16T08:18:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 -- 172.21.15.32:0/3710645413 learned_addr learned my addr 172.21.15.32:0/3710645413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a1161640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c153540 0x7fc19c13afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc19bfff640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c0cb6e0 0x7fc19c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 -- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c0cb6e0 msgr2=0x7fc19c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c0cb6e0 0x7fc19c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 -- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c153540 msgr2=0x7fc19c13afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c153540 0x7fc19c13afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 -- 172.21.15.32:0/3710645413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19400bd40 con 0x7fc19c0afaa0 2023-10-16T08:18:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a0960640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 0x7fc19c0b3b60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc19000d620 tx=0x7fc19000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000b620 con 0x7fc19c0afaa0 2023-10-16T08:18:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19000b780 con 0x7fc19c0afaa0 2023-10-16T08:18:40.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000bb10 con 0x7fc19c0afaa0 2023-10-16T08:18:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c13b4e0 con 0x7fc19c0afaa0 2023-10-16T08:18:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c0b2020 con 0x7fc19c0afaa0 2023-10-16T08:18:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.381+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c04e170 con 0x7fc19c0afaa0 2023-10-16T08:18:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.385+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc19000bc70 con 0x7fc19c0afaa0 2023-10-16T08:18:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.385+0000 7fc1997fa640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1800664a0 0x7fc180068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.385+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5663+0+0 (secure 0 0 0) 0x7fc190084200 con 0x7fc19c0afaa0 2023-10-16T08:18:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.385+0000 7fc19bfff640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1800664a0 0x7fc180068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.385+0000 7fc19bfff640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1800664a0 0x7fc180068960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc19c14e900 tx=0x7fc188007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.389+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc190051740 con 0x7fc19c0afaa0 2023-10-16T08:18:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:40.557+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc19c0cc730 con 0x7fc19c0afaa0 2023-10-16T08:18:41.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.193+0000 7fc1997fa640 1 -- 172.21.15.32:0/3710645413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v46) v1 ==== 160+0+0 (secure 0 0 0) 0x7fc190054d60 con 0x7fc19c0afaa0 2023-10-16T08:18:41.199 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'data_new' removed 2023-10-16T08:18:41.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.193+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1800664a0 msgr2=0x7fc180068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.193+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1800664a0 0x7fc180068960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc19c14e900 tx=0x7fc188007c90 comp rx=0 tx=0).stop 2023-10-16T08:18:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.193+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 msgr2=0x7fc19c0b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.193+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 0x7fc19c0b3b60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc19000d620 tx=0x7fc19000dae0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 shutdown_connections 2023-10-16T08:18:41.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc19c153540 0x7fc19c13afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc1800664a0 0x7fc180068960 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc19c0cb6e0 0x7fc19c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 --2- 172.21.15.32:0/3710645413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c0afaa0 0x7fc19c0b3b60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 >> 172.21.15.32:0/3710645413 conn(0x7fc19c06a820 msgr2=0x7fc19c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:41.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 shutdown_connections 2023-10-16T08:18:41.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.197+0000 7fc1a2bc4640 1 -- 172.21.15.32:0/3710645413 wait complete. 2023-10-16T08:18:41.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T08:18:41.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 Processor -- start 2023-10-16T08:18:41.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 -- start start 2023-10-16T08:18:41.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcea415cba0 0x7fcea40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fcea40b84e0 0x7fcea40cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40d1670 con 0x7fcea40d0040 2023-10-16T08:18:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40d1460 con 0x7fcea415cba0 2023-10-16T08:18:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea999d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40d04b0 con 0x7fcea40b84e0 2023-10-16T08:18:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcea415cba0 0x7fcea40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcea415cba0 0x7fcea40d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56926/0 (socket says 172.21.15.32:56926) 2023-10-16T08:18:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea2ffd640 1 -- 172.21.15.32:0/3102895903 learned_addr learned my addr 172.21.15.32:0/3102895903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.365+0000 7fcea27fc640 1 -- 172.21.15.32:0/3102895903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798648763 0 0) 0x7fcea40d1670 con 0x7fcea40d0040 2023-10-16T08:18:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fce9bfff640 1 --2- 172.21.15.32:0/3102895903 >> v2:172.21.15.186:3300/0 conn(0x7fcea40b84e0 0x7fcea40cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea27fc640 1 -- 172.21.15.32:0/3102895903 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce90003530 con 0x7fcea40d0040 2023-10-16T08:18:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea2ffd640 1 -- 172.21.15.32:0/3102895903 >> v2:172.21.15.186:3300/0 conn(0x7fcea40b84e0 msgr2=0x7fcea40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea2ffd640 1 --2- 172.21.15.32:0/3102895903 >> v2:172.21.15.186:3300/0 conn(0x7fcea40b84e0 0x7fcea40cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea2ffd640 1 -- 172.21.15.32:0/3102895903 >> v1:172.21.15.32:6789/0 conn(0x7fcea40d0040 legacy=0x7fcea40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea2ffd640 1 -- 172.21.15.32:0/3102895903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40d1950 con 0x7fcea415cba0 2023-10-16T08:18:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fce9bfff640 1 --2- 172.21.15.32:0/3102895903 >> v2:172.21.15.186:3300/0 conn(0x7fcea40b84e0 0x7fcea40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:41.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea2ffd640 1 --2- 172.21.15.32:0/3102895903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea415cba0 0x7fcea40d0f20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcea40cf2f0 tx=0x7fce9401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933014de269604b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:41.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea27fc640 1 -- 172.21.15.32:0/3102895903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940a8050 con 0x7fcea415cba0 2023-10-16T08:18:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea27fc640 1 -- 172.21.15.32:0/3102895903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940ac5f0 con 0x7fcea415cba0 2023-10-16T08:18:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea27fc640 1 -- 172.21.15.32:0/3102895903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940ac980 con 0x7fcea415cba0 2023-10-16T08:18:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 -- 172.21.15.32:0/3102895903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea415cba0 msgr2=0x7fcea40d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 --2- 172.21.15.32:0/3102895903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea415cba0 0x7fcea40d0f20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcea40cf2f0 tx=0x7fce9401c470 comp rx=0 tx=0).stop 2023-10-16T08:18:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 -- 172.21.15.32:0/3102895903 shutdown_connections 2023-10-16T08:18:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 --2- 172.21.15.32:0/3102895903 >> v2:172.21.15.186:3300/0 conn(0x7fcea40b84e0 0x7fcea40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 --2- 172.21.15.32:0/3102895903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea415cba0 0x7fcea40d0f20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 -- 172.21.15.32:0/3102895903 >> 172.21.15.32:0/3102895903 conn(0x7fcea40719c0 msgr2=0x7fcea4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.369+0000 7fcea999d640 1 -- 172.21.15.32:0/3102895903 shutdown_connections 2023-10-16T08:18:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 -- 172.21.15.32:0/3102895903 wait complete. 2023-10-16T08:18:41.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 Processor -- start 2023-10-16T08:18:41.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 -- start start 2023-10-16T08:18:41.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea40b84e0 0x7fcea414dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 0x7fcea4158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcea415cba0 0x7fcea41563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea414e230 con 0x7fcea40d0040 2023-10-16T08:18:41.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4151d20 con 0x7fcea40b84e0 2023-10-16T08:18:41.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea999d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4152000 con 0x7fcea415cba0 2023-10-16T08:18:41.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 0x7fcea4158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 0x7fcea4158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56936/0 (socket says 172.21.15.32:56936) 2023-10-16T08:18:41.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 -- 172.21.15.32:0/4274204645 learned_addr learned my addr 172.21.15.32:0/4274204645 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:41.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea2ffd640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea40b84e0 0x7fcea414dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea37fe640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcea415cba0 0x7fcea41563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 -- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcea415cba0 msgr2=0x7fcea41563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcea415cba0 0x7fcea41563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 -- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea40b84e0 msgr2=0x7fcea414dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea40b84e0 0x7fcea414dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 -- 172.21.15.32:0/4274204645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940094d0 con 0x7fcea40d0040 2023-10-16T08:18:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea37fe640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcea415cba0 0x7fcea41563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fcea2ffd640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea40b84e0 0x7fcea414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.373+0000 7fce9bfff640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 0x7fcea4158f00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c7c0 tx=0x7fce8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c007a30 con 0x7fcea40d0040 2023-10-16T08:18:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c007b90 con 0x7fcea40d0040 2023-10-16T08:18:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea41522e0 con 0x7fcea40d0040 2023-10-16T08:18:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00ce70 con 0x7fcea40d0040 2023-10-16T08:18:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea41528a0 con 0x7fcea40d0040 2023-10-16T08:18:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea414e3e0 con 0x7fcea40d0040 2023-10-16T08:18:41.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce8c0113f0 con 0x7fcea40d0040 2023-10-16T08:18:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea0ff9640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fce840664a0 0x7fce84068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fce8c0841e0 con 0x7fcea40d0040 2023-10-16T08:18:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.377+0000 7fcea2ffd640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fce840664a0 0x7fce84068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.381+0000 7fcea2ffd640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fce840664a0 0x7fce84068960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fce940ae000 tx=0x7fce940a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:41.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.381+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce8c051780 con 0x7fcea40d0040 2023-10-16T08:18:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.537+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcea404e170 con 0x7fcea40d0040 2023-10-16T08:18:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.541+0000 7fcea0ff9640 1 -- 172.21.15.32:0/4274204645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fce8c054da0 con 0x7fcea40d0040 2023-10-16T08:18:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fce840664a0 msgr2=0x7fce84068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fce840664a0 0x7fce84068960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fce940ae000 tx=0x7fce940a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 msgr2=0x7fcea4158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 0x7fcea4158f00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c7c0 tx=0x7fce8c00cc80 comp rx=0 tx=0).stop 2023-10-16T08:18:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 shutdown_connections 2023-10-16T08:18:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fce840664a0 0x7fce84068960 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcea415cba0 0x7fcea41563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea40d0040 0x7fcea4158f00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 --2- 172.21.15.32:0/4274204645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea40b84e0 0x7fcea414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 >> 172.21.15.32:0/4274204645 conn(0x7fcea40719c0 msgr2=0x7fcea4159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 shutdown_connections 2023-10-16T08:18:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.545+0000 7fcea999d640 1 -- 172.21.15.32:0/4274204645 wait complete. 2023-10-16T08:18:41.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-10-16T08:18:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abe43f640 1 Processor -- start 2023-10-16T08:18:41.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abe43f640 1 -- start start 2023-10-16T08:18:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abe43f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 0x7f0ab80ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abe43f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80723d0 con 0x7f0ab8159ba0 2023-10-16T08:18:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abe43f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80717c0 con 0x7f0ab80adc10 2023-10-16T08:18:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abe43f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8071aa0 con 0x7f0ab81597c0 2023-10-16T08:18:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0ab8159ba0 0x7f0ab80cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49340/0 (socket says 172.21.15.32:49340) 2023-10-16T08:18:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3262241812 learned_addr learned my addr 172.21.15.32:0/3262241812 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab7fff640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 0x7f0ab80ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802231793 0 0) 0x7f0ab80723d0 con 0x7f0ab8159ba0 2023-10-16T08:18:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003530 con 0x7f0ab8159ba0 2023-10-16T08:18:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955618905 0 0) 0x7f0ab80717c0 con 0x7f0ab80adc10 2023-10-16T08:18:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab80723d0 con 0x7f0ab80adc10 2023-10-16T08:18:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4122260419 0 0) 0x7f0aa0003530 con 0x7f0ab8159ba0 2023-10-16T08:18:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 msgr2=0x7f0ab80ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 0x7f0ab80ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 >> v1:172.21.15.186:6789/0 conn(0x7f0ab80adc10 legacy=0x7f0ab815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8071d80 con 0x7f0ab8159ba0 2023-10-16T08:18:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f0aac001680 con 0x7f0ab8159ba0 2023-10-16T08:18:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 >> v1:172.21.15.32:6789/0 conn(0x7f0ab8159ba0 legacy=0x7f0ab80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa0003b40 0x7f0aa0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa00047d0 0x7f0aa0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 0x7f0aa0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8071aa0 con 0x7f0aa00047d0 2023-10-16T08:18:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000ffa0 con 0x7f0aa0003b40 2023-10-16T08:18:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00100e0 con 0x7f0ab81597c0 2023-10-16T08:18:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa00047d0 0x7f0aa0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab7fff640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa0003b40 0x7f0aa0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0abc9dc640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 0x7f0aa0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 msgr2=0x7f0aa0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab81597c0 0x7f0aa0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa0003b40 msgr2=0x7f0aa0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa0003b40 0x7f0aa0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.713+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3262241812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0003530 con 0x7f0aa00047d0 2023-10-16T08:18:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa00047d0 0x7f0aa0004c50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00c160 tx=0x7f0aac01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e320f50b871d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0058e0 con 0x7f0aa00047d0 2023-10-16T08:18:41.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac005a60 con 0x7f0aa00047d0 2023-10-16T08:18:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0ab6ffd640 1 -- 172.21.15.32:0/3262241812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac005e10 con 0x7f0aa00047d0 2023-10-16T08:18:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abe43f640 1 -- 172.21.15.32:0/3262241812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa00047d0 msgr2=0x7f0aa0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3262241812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa00047d0 0x7f0aa0004c50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00c160 tx=0x7f0aac01cba0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abc9dc640 1 -- 172.21.15.32:0/3262241812 reap_dead start 2023-10-16T08:18:41.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abe43f640 1 -- 172.21.15.32:0/3262241812 shutdown_connections 2023-10-16T08:18:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abe43f640 1 -- 172.21.15.32:0/3262241812 >> 172.21.15.32:0/3262241812 conn(0x7f0ab806c190 msgr2=0x7f0ab806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abe43f640 1 -- 172.21.15.32:0/3262241812 shutdown_connections 2023-10-16T08:18:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.717+0000 7f0abe43f640 1 -- 172.21.15.32:0/3262241812 wait complete. 2023-10-16T08:18:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 Processor -- start 2023-10-16T08:18:41.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 -- start start 2023-10-16T08:18:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa00047d0 0x7f0ab814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 0x7f0ab813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab8155c50 0x7f0ab813fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab814fbb0 con 0x7f0ab81557f0 2023-10-16T08:18:41.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab814fe90 con 0x7f0aa00047d0 2023-10-16T08:18:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8150170 con 0x7f0ab8155c50 2023-10-16T08:18:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 0x7f0ab813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 0x7f0ab813f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56958/0 (socket says 172.21.15.32:56958) 2023-10-16T08:18:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3201828947 learned_addr learned my addr 172.21.15.32:0/3201828947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:41.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abc9dc640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab8155c50 0x7f0ab813fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab7fff640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa00047d0 0x7f0ab814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab8155c50 msgr2=0x7f0ab813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab8155c50 0x7f0ab813fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa00047d0 msgr2=0x7f0ab814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa00047d0 0x7f0ab814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 -- 172.21.15.32:0/3201828947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8150450 con 0x7f0ab81557f0 2023-10-16T08:18:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab77fe640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 0x7f0ab813f8b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0ab040 tx=0x7f0aa0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0a5cb0 con 0x7f0ab81557f0 2023-10-16T08:18:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac0a5e10 con 0x7f0ab81557f0 2023-10-16T08:18:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0049f0 con 0x7f0ab81557f0 2023-10-16T08:18:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab8150730 con 0x7f0ab81557f0 2023-10-16T08:18:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.721+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab8150c50 con 0x7f0ab81557f0 2023-10-16T08:18:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.725+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab8074010 con 0x7f0ab81557f0 2023-10-16T08:18:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.725+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0aac004b50 con 0x7f0ab81557f0 2023-10-16T08:18:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.725+0000 7f0ab4ff9640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0aa40664d0 0x7f0aa4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.725+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0aac11d780 con 0x7f0ab81557f0 2023-10-16T08:18:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.725+0000 7f0ab7fff640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0aa40664d0 0x7f0aa4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.725+0000 7f0ab7fff640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0aa40664d0 0x7f0aa4068990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009a80 tx=0x7f0aa8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.729+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0aac0ead40 con 0x7f0ab81557f0 2023-10-16T08:18:41.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:41.949+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f0ab80ad9f0 con 0x7f0ab81557f0 2023-10-16T08:18:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.201+0000 7f0ab4ff9640 1 -- 172.21.15.32:0/3201828947 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v153) v1 ==== 147+0+0 (secure 0 0 0) 0x7f0aac0be0d0 con 0x7f0ab81557f0 2023-10-16T08:18:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0aa40664d0 msgr2=0x7f0aa4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0aa40664d0 0x7f0aa4068990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009a80 tx=0x7f0aa8009190 comp rx=0 tx=0).stop 2023-10-16T08:18:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 msgr2=0x7f0ab813f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 0x7f0ab813f8b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0ab040 tx=0x7f0aa0005800 comp rx=0 tx=0).stop 2023-10-16T08:18:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 shutdown_connections 2023-10-16T08:18:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0aa40664d0 0x7f0aa4068990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0ab8155c50 0x7f0ab813fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab81557f0 0x7f0ab813f8b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 --2- 172.21.15.32:0/3201828947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0aa00047d0 0x7f0ab814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 >> 172.21.15.32:0/3201828947 conn(0x7f0ab806c190 msgr2=0x7f0ab8151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 shutdown_connections 2023-10-16T08:18:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.205+0000 7f0abe43f640 1 -- 172.21.15.32:0/3201828947 wait complete. 2023-10-16T08:18:42.231 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2023-10-16T08:18:42.231 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T08:18:42.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:42.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 Processor -- start 2023-10-16T08:18:42.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 -- start start 2023-10-16T08:18:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa1340ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1340b07f0 0x7fa1340cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1340c96f0 con 0x7fa1340ad960 2023-10-16T08:18:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa134164900 con 0x7fa1340b07f0 2023-10-16T08:18:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa13a9aa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340cb950 con 0x7fa134164520 2023-10-16T08:18:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa138f47640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1340b07f0 0x7fa1340cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa133fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa1340ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa138f47640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fa1340b07f0 0x7fa1340cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:53852/0 (socket says 172.21.15.32:53852) 2023-10-16T08:18:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.369+0000 7fa138f47640 1 -- 172.21.15.32:0/2077412434 learned_addr learned my addr 172.21.15.32:0/2077412434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa138f47640 1 -- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 msgr2=0x7fa1340ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa138f47640 1 --2- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa1340ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa138f47640 1 -- 172.21.15.32:0/2077412434 >> v1:172.21.15.186:6789/0 conn(0x7fa1340ad960 legacy=0x7fa1340b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa138f47640 1 -- 172.21.15.32:0/2077412434 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1340cbed0 con 0x7fa1340b07f0 2023-10-16T08:18:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa138f47640 1 --2- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340b07f0 0x7fa1340cc870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e50 tx=0x7fa12c01c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab5f61d558ba0ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa132ffd640 1 -- 172.21.15.32:0/2077412434 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0a58d0 con 0x7fa1340b07f0 2023-10-16T08:18:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa132ffd640 1 -- 172.21.15.32:0/2077412434 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c0a5a30 con 0x7fa1340b07f0 2023-10-16T08:18:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa132ffd640 1 -- 172.21.15.32:0/2077412434 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0a5dc0 con 0x7fa1340b07f0 2023-10-16T08:18:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340b07f0 msgr2=0x7fa1340cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.373+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340b07f0 0x7fa1340cc870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e50 tx=0x7fa12c01c9e0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2077412434 shutdown_connections 2023-10-16T08:18:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340b07f0 0x7fa1340cc870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2077412434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa1340ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2077412434 >> 172.21.15.32:0/2077412434 conn(0x7fa13406c190 msgr2=0x7fa13406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2077412434 shutdown_connections 2023-10-16T08:18:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2077412434 wait complete. 2023-10-16T08:18:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 Processor -- start 2023-10-16T08:18:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- start start 2023-10-16T08:18:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340ad960 0x7fa1341484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 0x7fa13415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa13415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c3870 con 0x7fa1340b07f0 2023-10-16T08:18:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c3b50 con 0x7fa1340ad960 2023-10-16T08:18:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa13a9aa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c3e30 con 0x7fa134164520 2023-10-16T08:18:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 0x7fa13415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 0x7fa13415eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56972/0 (socket says 172.21.15.32:56972) 2023-10-16T08:18:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 -- 172.21.15.32:0/2031656314 learned_addr learned my addr 172.21.15.32:0/2031656314 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa138f47640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa13415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa133fff640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340ad960 0x7fa1341484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 -- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 msgr2=0x7fa13415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa13415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 -- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340ad960 msgr2=0x7fa1341484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340ad960 0x7fa1341484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa1337fe640 1 -- 172.21.15.32:0/2031656314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c009b30 con 0x7fa1340b07f0 2023-10-16T08:18:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.377+0000 7fa133fff640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340ad960 0x7fa1341484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa1337fe640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 0x7fa13415eb80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c3c0 tx=0x7fa12800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280052d0 con 0x7fa1340b07f0 2023-10-16T08:18:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa128005430 con 0x7fa1340b07f0 2023-10-16T08:18:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128003840 con 0x7fa1340b07f0 2023-10-16T08:18:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1340c4110 con 0x7fa1340b07f0 2023-10-16T08:18:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa134148a20 con 0x7fa1340b07f0 2023-10-16T08:18:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134074010 con 0x7fa1340b07f0 2023-10-16T08:18:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa1280039a0 con 0x7fa1340b07f0 2023-10-16T08:18:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa130ff9640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa1200664a0 0x7fa120068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa128083130 con 0x7fa1340b07f0 2023-10-16T08:18:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.381+0000 7fa133fff640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa1200664a0 0x7fa120068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.385+0000 7fa133fff640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa1200664a0 0x7fa120068960 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa124009f50 tx=0x7fa124009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.385+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa128050820 con 0x7fa1340b07f0 2023-10-16T08:18:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1340c45c0 con 0x7fa1340b07f0 2023-10-16T08:18:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa130ff9640 1 -- 172.21.15.32:0/2031656314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fa128053e40 con 0x7fa1340b07f0 2023-10-16T08:18:42.551 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:42.551 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:41.196439+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/1556343693":"2023-10-17T08:18:37.187636+0000","172.21.15.32:6834/1556343693":"2023-10-17T08:18:37.187636+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:42.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa1200664a0 msgr2=0x7fa120068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa1200664a0 0x7fa120068960 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa124009f50 tx=0x7fa124009190 comp rx=0 tx=0).stop 2023-10-16T08:18:42.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 msgr2=0x7fa13415eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 0x7fa13415eb80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c3c0 tx=0x7fa12800c880 comp rx=0 tx=0).stop 2023-10-16T08:18:42.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 shutdown_connections 2023-10-16T08:18:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa1200664a0 0x7fa120068960 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa134164520 0x7fa13415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1340b07f0 0x7fa13415eb80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 --2- 172.21.15.32:0/2031656314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1340ad960 0x7fa1341484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 >> 172.21.15.32:0/2031656314 conn(0x7fa13406c190 msgr2=0x7fa1340ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 shutdown_connections 2023-10-16T08:18:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.545+0000 7fa13a9aa640 1 -- 172.21.15.32:0/2031656314 wait complete. 2023-10-16T08:18:42.577 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:42.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:42.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.717+0000 7fc0116bb640 1 Processor -- start 2023-10-16T08:18:42.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.717+0000 7fc0116bb640 1 -- start start 2023-10-16T08:18:42.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc0116bb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc00c0cf210 0x7fc00c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc0116bb640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc00c0d3320 0x7fc00c0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc0116bb640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c0cfb30 con 0x7fc00c15cdc0 2023-10-16T08:18:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc0116bb640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c0cea10 con 0x7fc00c0cf210 2023-10-16T08:18:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc0116bb640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c0d0880 con 0x7fc00c0d3320 2023-10-16T08:18:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc00c0cf210 0x7fc00c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc00c0cf210 0x7fc00c0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56988/0 (socket says 172.21.15.32:56988) 2023-10-16T08:18:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 -- 172.21.15.32:0/157096705 learned_addr learned my addr 172.21.15.32:0/157096705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc010eba640 1 --2- 172.21.15.32:0/157096705 >> v2:172.21.15.186:3300/0 conn(0x7fc00c0d3320 0x7fc00c0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 -- 172.21.15.32:0/157096705 >> v2:172.21.15.186:3300/0 conn(0x7fc00c0d3320 msgr2=0x7fc00c0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 --2- 172.21.15.32:0/157096705 >> v2:172.21.15.186:3300/0 conn(0x7fc00c0d3320 0x7fc00c0ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 -- 172.21.15.32:0/157096705 >> v1:172.21.15.186:6789/0 conn(0x7fc00c15cdc0 legacy=0x7fc00c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.721+0000 7fc00bfff640 1 -- 172.21.15.32:0/157096705 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0b1c00 con 0x7fc00c0cf210 2023-10-16T08:18:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc00bfff640 1 --2- 172.21.15.32:0/157096705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c0cf210 0x7fc00c0cf5f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc00c08b980 tx=0x7fbffc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37374d574290ddbb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc00affd640 1 -- 172.21.15.32:0/157096705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0ab640 con 0x7fc00c0cf210 2023-10-16T08:18:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc00affd640 1 -- 172.21.15.32:0/157096705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc0a7070 con 0x7fc00c0cf210 2023-10-16T08:18:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc00affd640 1 -- 172.21.15.32:0/157096705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0ab9c0 con 0x7fc00c0cf210 2023-10-16T08:18:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 -- 172.21.15.32:0/157096705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c0cf210 msgr2=0x7fc00c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 --2- 172.21.15.32:0/157096705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c0cf210 0x7fc00c0cf5f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc00c08b980 tx=0x7fbffc01c440 comp rx=0 tx=0).stop 2023-10-16T08:18:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 -- 172.21.15.32:0/157096705 shutdown_connections 2023-10-16T08:18:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 --2- 172.21.15.32:0/157096705 >> v2:172.21.15.186:3300/0 conn(0x7fc00c0d3320 0x7fc00c0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 --2- 172.21.15.32:0/157096705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c0cf210 0x7fc00c0cf5f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 -- 172.21.15.32:0/157096705 >> 172.21.15.32:0/157096705 conn(0x7fc00c0d2800 msgr2=0x7fc00c0d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 -- 172.21.15.32:0/157096705 shutdown_connections 2023-10-16T08:18:42.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 -- 172.21.15.32:0/157096705 wait complete. 2023-10-16T08:18:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 Processor -- start 2023-10-16T08:18:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.725+0000 7fc0116bb640 1 -- start start 2023-10-16T08:18:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c0cf210 0x7fc00c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc00c0d3320 0x7fc00c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 0x7fc00c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0cc2a0 con 0x7fc00c15cdc0 2023-10-16T08:18:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0cbfc0 con 0x7fc00c0cf210 2023-10-16T08:18:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c155710 con 0x7fc00c0d3320 2023-10-16T08:18:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 0x7fc00c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 0x7fc00c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56990/0 (socket says 172.21.15.32:56990) 2023-10-16T08:18:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 -- 172.21.15.32:0/1378035471 learned_addr learned my addr 172.21.15.32:0/1378035471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc00bfff640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c0cf210 0x7fc00c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc00b7fe640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc00c0d3320 0x7fc00c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 -- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc00c0d3320 msgr2=0x7fc00c0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc00c0d3320 0x7fc00c0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 -- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c0cf210 msgr2=0x7fc00c1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c0cf210 0x7fc00c1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 -- 172.21.15.32:0/1378035471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009480 con 0x7fc00c15cdc0 2023-10-16T08:18:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc010eba640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 0x7fc00c0cb620 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc00400efc0 tx=0x7fc00400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004014480 con 0x7fc00c15cdc0 2023-10-16T08:18:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400f040 con 0x7fc00c15cdc0 2023-10-16T08:18:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004014820 con 0x7fc00c15cdc0 2023-10-16T08:18:42.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c1559f0 con 0x7fc00c15cdc0 2023-10-16T08:18:42.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c156730 con 0x7fc00c15cdc0 2023-10-16T08:18:42.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c155570 con 0x7fc00c15cdc0 2023-10-16T08:18:42.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.729+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc004014980 con 0x7fc00c15cdc0 2023-10-16T08:18:42.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.733+0000 7fc008ff9640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbff80664a0 0x7fbff8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.733+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc004084740 con 0x7fc00c15cdc0 2023-10-16T08:18:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.733+0000 7fc00bfff640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbff80664a0 0x7fbff8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.733+0000 7fc00bfff640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbff80664a0 0x7fbff8068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0af000 tx=0x7fbffc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.737+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc00c155570 con 0x7fc00c15cdc0 2023-10-16T08:18:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc00c155570 con 0x7fc00c15cdc0 2023-10-16T08:18:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc008ff9640 1 -- 172.21.15.32:0/1378035471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc00c155570 con 0x7fc00c15cdc0 2023-10-16T08:18:42.899 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbff80664a0 msgr2=0x7fbff8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbff80664a0 0x7fbff8068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0af000 tx=0x7fbffc0a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 msgr2=0x7fc00c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 0x7fc00c0cb620 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc00400efc0 tx=0x7fc00400c490 comp rx=0 tx=0).stop 2023-10-16T08:18:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 shutdown_connections 2023-10-16T08:18:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c15cdc0 0x7fc00c0cb620 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbff80664a0 0x7fbff8068960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc00c0d3320 0x7fc00c0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 --2- 172.21.15.32:0/1378035471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c0cf210 0x7fc00c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 >> 172.21.15.32:0/1378035471 conn(0x7fc00c0d2800 msgr2=0x7fc00c0b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 shutdown_connections 2023-10-16T08:18:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:42.893+0000 7fc0116bb640 1 -- 172.21.15.32:0/1378035471 wait complete. 2023-10-16T08:18:42.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-10-16T08:18:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.057+0000 7fafe759e640 1 Processor -- start 2023-10-16T08:18:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.057+0000 7fafe759e640 1 -- start start 2023-10-16T08:18:43.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafe815c980 0x7fafe806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:43.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe759e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:43.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe759e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe80d3510 con 0x7fafe80d0d80 2023-10-16T08:18:43.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe759e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe80cd2b0 con 0x7fafe815c980 2023-10-16T08:18:43.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe759e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe80d2f50 con 0x7fafe815cd60 2023-10-16T08:18:43.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafe815c980 0x7fafe806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:43.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafe815c980 0x7fafe806af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55456/0 (socket says 172.21.15.32:55456) 2023-10-16T08:18:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 -- 172.21.15.32:0/1351191185 learned_addr learned my addr 172.21.15.32:0/1351191185 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 -- 172.21.15.32:0/1351191185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 msgr2=0x7fafe80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 --2- 172.21.15.32:0/1351191185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 -- 172.21.15.32:0/1351191185 >> v1:172.21.15.186:6789/0 conn(0x7fafe80d0d80 legacy=0x7fafe80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.061+0000 7fafe659c640 1 -- 172.21.15.32:0/1351191185 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe80d3230 con 0x7fafe815c980 2023-10-16T08:18:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe659c640 1 --2- 172.21.15.32:0/1351191185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe806af90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009860 tx=0x7fafd801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0dc609e05281dfd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:43.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe5d9b640 1 -- 172.21.15.32:0/1351191185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80ac410 con 0x7fafe815c980 2023-10-16T08:18:43.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe5d9b640 1 -- 172.21.15.32:0/1351191185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd80ac570 con 0x7fafe815c980 2023-10-16T08:18:43.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe5d9b640 1 -- 172.21.15.32:0/1351191185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80ac900 con 0x7fafe815c980 2023-10-16T08:18:43.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 -- 172.21.15.32:0/1351191185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 msgr2=0x7fafe806af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:43.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 --2- 172.21.15.32:0/1351191185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe806af90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009860 tx=0x7fafd801c4e0 comp rx=0 tx=0).stop 2023-10-16T08:18:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 -- 172.21.15.32:0/1351191185 shutdown_connections 2023-10-16T08:18:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 --2- 172.21.15.32:0/1351191185 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 --2- 172.21.15.32:0/1351191185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe806af90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 -- 172.21.15.32:0/1351191185 >> 172.21.15.32:0/1351191185 conn(0x7fafe809ba80 msgr2=0x7fafe80d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 -- 172.21.15.32:0/1351191185 shutdown_connections 2023-10-16T08:18:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 -- 172.21.15.32:0/1351191185 wait complete. 2023-10-16T08:18:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 Processor -- start 2023-10-16T08:18:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.065+0000 7fafe759e640 1 -- start start 2023-10-16T08:18:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafe80d0d80 0x7fafe80b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe80b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe8156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8153720 con 0x7fafe815c980 2023-10-16T08:18:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe815ffd0 con 0x7fafe80d0d80 2023-10-16T08:18:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe815fcf0 con 0x7fafe815cd60 2023-10-16T08:18:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe80b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:43.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe80b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55462/0 (socket says 172.21.15.32:55462) 2023-10-16T08:18:43.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 -- 172.21.15.32:0/558937516 learned_addr learned my addr 172.21.15.32:0/558937516 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:43.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe659c640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafe80d0d80 0x7fafe80b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:43.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe6d9d640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe8156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:43.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 -- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 msgr2=0x7fafe8156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:43.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe8156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:43.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 -- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafe80d0d80 msgr2=0x7fafe80b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:43.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafe80d0d80 0x7fafe80b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:43.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 -- 172.21.15.32:0/558937516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd8009470 con 0x7fafe815c980 2023-10-16T08:18:43.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe6d9d640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe8156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:43.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdffff640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe80b0900 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fafd000b410 tx=0x7fafd000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:43.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0011830 con 0x7fafe815c980 2023-10-16T08:18:43.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0011990 con 0x7fafe815c980 2023-10-16T08:18:43.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0011d40 con 0x7fafe815c980 2023-10-16T08:18:43.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe81530d0 con 0x7fafe815c980 2023-10-16T08:18:43.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe8157ae0 con 0x7fafe815c980 2023-10-16T08:18:43.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe81538d0 con 0x7fafe815c980 2023-10-16T08:18:43.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.069+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fafd001a400 con 0x7fafe815c980 2023-10-16T08:18:43.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.073+0000 7fafdeffd640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fafc80664a0 0x7fafc8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:43.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.073+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fafd0083d10 con 0x7fafe815c980 2023-10-16T08:18:43.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.073+0000 7fafe659c640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fafc80664a0 0x7fafc8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:43.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.073+0000 7fafe659c640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fafc80664a0 0x7fafc8068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafd80ae000 tx=0x7fafd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:43.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.077+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fafe81538d0 con 0x7fafe815c980 2023-10-16T08:18:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:43.293+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fafe804e9c0 con 0x7fafe815c980 2023-10-16T08:18:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafdeffd640 1 -- 172.21.15.32:0/558937516 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v155) v1 ==== 137+0+0 (secure 0 0 0) 0x7fafd00150d0 con 0x7fafe815c980 2023-10-16T08:18:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fafc80664a0 msgr2=0x7fafc8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fafc80664a0 0x7fafc8068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafd80ae000 tx=0x7fafd8008040 comp rx=0 tx=0).stop 2023-10-16T08:18:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 msgr2=0x7fafe80b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe80b0900 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fafd000b410 tx=0x7fafd000b8d0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 shutdown_connections 2023-10-16T08:18:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fafc80664a0 0x7fafc8068960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fafe815cd60 0x7fafe8156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafe815c980 0x7fafe80b0900 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 --2- 172.21.15.32:0/558937516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafe80d0d80 0x7fafe80b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 >> 172.21.15.32:0/558937516 conn(0x7fafe809ba80 msgr2=0x7fafe81553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.213+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 shutdown_connections 2023-10-16T08:18:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.217+0000 7fafe759e640 1 -- 172.21.15.32:0/558937516 wait complete. 2023-10-16T08:18:44.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T08:18:44.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.389+0000 7f1c5de6b640 1 Processor -- start 2023-10-16T08:18:44.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.389+0000 7f1c5de6b640 1 -- start start 2023-10-16T08:18:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c5de6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1c580ca980 0x7f1c580b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c5de6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c580b3100 0x7f1c580b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c5de6b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5806d500 con 0x7f1c580b2130 2023-10-16T08:18:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c5de6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c580b2580 con 0x7f1c580b3100 2023-10-16T08:18:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c5de6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c58159f20 con 0x7f1c580ca980 2023-10-16T08:18:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c56ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c580b3100 0x7f1c580b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c57fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c580b2130 0x7f1c580cc120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37220/0 (socket says 172.21.15.32:37220) 2023-10-16T08:18:44.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c57fff640 1 -- 172.21.15.32:0/1624600155 learned_addr learned my addr 172.21.15.32:0/1624600155 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c56ffd640 1 --2- 172.21.15.32:0/1624600155 >> v2:172.21.15.32:3300/0 conn(0x7f1c580b3100 0x7f1c580b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55466/0 (socket says 172.21.15.32:55466) 2023-10-16T08:18:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c577fe640 1 --2- 172.21.15.32:0/1624600155 >> v2:172.21.15.186:3300/0 conn(0x7f1c580ca980 0x7f1c580b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c56ffd640 1 -- 172.21.15.32:0/1624600155 >> v2:172.21.15.186:3300/0 conn(0x7f1c580ca980 msgr2=0x7f1c580b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c56ffd640 1 --2- 172.21.15.32:0/1624600155 >> v2:172.21.15.186:3300/0 conn(0x7f1c580ca980 0x7f1c580b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c56ffd640 1 -- 172.21.15.32:0/1624600155 >> v1:172.21.15.32:6789/0 conn(0x7f1c580b2130 legacy=0x7f1c580cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c56ffd640 1 -- 172.21.15.32:0/1624600155 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5806c620 con 0x7f1c580b3100 2023-10-16T08:18:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c567fc640 1 -- 172.21.15.32:0/1624600155 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713117186 0 0) 0x7f1c5806d500 con 0x7f1c580b2130 2023-10-16T08:18:44.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.393+0000 7f1c577fe640 1 --2- 172.21.15.32:0/1624600155 >> v2:172.21.15.186:3300/0 conn(0x7f1c580ca980 0x7f1c580b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:44.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c56ffd640 1 --2- 172.21.15.32:0/1624600155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c580b1bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1c580ce4b0 tx=0x7f1c4c01eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a4990505e492fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c567fc640 1 -- 172.21.15.32:0/1624600155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0a7bb0 con 0x7f1c580b3100 2023-10-16T08:18:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c567fc640 1 -- 172.21.15.32:0/1624600155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c0a7d10 con 0x7f1c580b3100 2023-10-16T08:18:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c567fc640 1 -- 172.21.15.32:0/1624600155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0af660 con 0x7f1c580b3100 2023-10-16T08:18:44.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 -- 172.21.15.32:0/1624600155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 msgr2=0x7f1c580b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/1624600155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c580b1bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1c580ce4b0 tx=0x7f1c4c01eae0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 -- 172.21.15.32:0/1624600155 shutdown_connections 2023-10-16T08:18:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/1624600155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c580b1bf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/1624600155 >> v2:172.21.15.186:3300/0 conn(0x7f1c580ca980 0x7f1c580b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 -- 172.21.15.32:0/1624600155 >> 172.21.15.32:0/1624600155 conn(0x7f1c5806a490 msgr2=0x7f1c580ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 -- 172.21.15.32:0/1624600155 shutdown_connections 2023-10-16T08:18:44.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 -- 172.21.15.32:0/1624600155 wait complete. 2023-10-16T08:18:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 Processor -- start 2023-10-16T08:18:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.397+0000 7f1c5de6b640 1 -- start start 2023-10-16T08:18:44.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c580b2130 0x7f1c58152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c5815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c580ca980 0x7f1c581580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c580ac9c0 con 0x7f1c580b3100 2023-10-16T08:18:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58157c60 con 0x7f1c580b2130 2023-10-16T08:18:44.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58157670 con 0x7f1c580ca980 2023-10-16T08:18:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c5815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c5815a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55472/0 (socket says 172.21.15.32:55472) 2023-10-16T08:18:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 -- 172.21.15.32:0/98065445 learned_addr learned my addr 172.21.15.32:0/98065445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 -- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c580ca980 msgr2=0x7f1c581580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c580ca980 0x7f1c581580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 -- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c580b2130 msgr2=0x7f1c58152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c580b2130 0x7f1c58152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 -- 172.21.15.32:0/98065445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c00b6c0 con 0x7f1c580b3100 2023-10-16T08:18:44.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c56ffd640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c5815a060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b2040 tx=0x7f1c4c01b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0a7bb0 con 0x7f1c580b3100 2023-10-16T08:18:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c0b8460 con 0x7f1c580b3100 2023-10-16T08:18:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c580ab260 con 0x7f1c580b3100 2023-10-16T08:18:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0b8830 con 0x7f1c580b3100 2023-10-16T08:18:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c58152970 con 0x7f1c580b3100 2023-10-16T08:18:44.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c58073fa0 con 0x7f1c580b3100 2023-10-16T08:18:44.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.401+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c4c0b6070 con 0x7f1c580b3100 2023-10-16T08:18:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.405+0000 7f1c3ffff640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1c440664a0 0x7f1c44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.405+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1c4c11eaf0 con 0x7f1c580b3100 2023-10-16T08:18:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.405+0000 7f1c577fe640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1c440664a0 0x7f1c44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.405+0000 7f1c577fe640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1c440664a0 0x7f1c44068960 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c48006fd0 tx=0x7f1c48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.409+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1c4c0ec130 con 0x7f1c580b3100 2023-10-16T08:18:44.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.565+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1c58157da0 con 0x7f1c580b3100 2023-10-16T08:18:44.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.569+0000 7f1c3ffff640 1 -- 172.21.15.32:0/98065445 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c58157da0 con 0x7f1c580b3100 2023-10-16T08:18:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1c440664a0 msgr2=0x7f1c44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1c440664a0 0x7f1c44068960 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c48006fd0 tx=0x7f1c48008040 comp rx=0 tx=0).stop 2023-10-16T08:18:44.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 msgr2=0x7f1c5815a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c5815a060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0b2040 tx=0x7f1c4c01b880 comp rx=0 tx=0).stop 2023-10-16T08:18:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 shutdown_connections 2023-10-16T08:18:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1c440664a0 0x7f1c44068960 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1c580ca980 0x7f1c581580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c580b3100 0x7f1c5815a060 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 --2- 172.21.15.32:0/98065445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c580b2130 0x7f1c58152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 >> 172.21.15.32:0/98065445 conn(0x7f1c5806a490 msgr2=0x7f1c58070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 shutdown_connections 2023-10-16T08:18:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.573+0000 7f1c5de6b640 1 -- 172.21.15.32:0/98065445 wait complete. 2023-10-16T08:18:44.598 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T08:18:44.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:18:44.687 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:18:44.750 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T08:18:44.751 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:44.751 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:44.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:18:44.761 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:18:44.761 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:44.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:18:44.771 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:44.771 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:44.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:18:44.810 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:18:44.810 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:44.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:18:44.824 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T08:18:44.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 Processor -- start 2023-10-16T08:18:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 -- start start 2023-10-16T08:18:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53ec15cb90 0x7f53ec0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec0d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f53ec0d1400 0x7f53ec0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0d1890 con 0x7f53ec15cb90 2023-10-16T08:18:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0ce770 con 0x7f53ec0d1400 2023-10-16T08:18:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f430f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0d0600 con 0x7f53ec0d3010 2023-10-16T08:18:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53ec15cb90 0x7f53ec0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53ec15cb90 0x7f53ec0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55480/0 (socket says 172.21.15.32:55480) 2023-10-16T08:18:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 -- 172.21.15.32:0/747248135 learned_addr learned my addr 172.21.15.32:0/747248135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f28ac640 1 --2- 172.21.15.32:0/747248135 >> v2:172.21.15.186:3300/0 conn(0x7f53ec0d1400 0x7f53ec0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f18aa640 1 --2- 172.21.15.32:0/747248135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec0d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 -- 172.21.15.32:0/747248135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 msgr2=0x7f53ec0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 --2- 172.21.15.32:0/747248135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec0d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 -- 172.21.15.32:0/747248135 >> v2:172.21.15.186:3300/0 conn(0x7f53ec0d1400 msgr2=0x7f53ec0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 --2- 172.21.15.32:0/747248135 >> v2:172.21.15.186:3300/0 conn(0x7f53ec0d1400 0x7f53ec0cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f20ab640 1 -- 172.21.15.32:0/747248135 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0c6450 con 0x7f53ec15cb90 2023-10-16T08:18:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.989+0000 7f53f28ac640 1 --2- 172.21.15.32:0/747248135 >> v2:172.21.15.186:3300/0 conn(0x7f53ec0d1400 0x7f53ec0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f20ab640 1 --2- 172.21.15.32:0/747248135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec15cb90 0x7f53ec0cfb10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0725c0 tx=0x7f53e801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88388b7a4ff7e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f10a9640 1 -- 172.21.15.32:0/747248135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80ab600 con 0x7f53ec15cb90 2023-10-16T08:18:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f10a9640 1 -- 172.21.15.32:0/747248135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80ab760 con 0x7f53ec15cb90 2023-10-16T08:18:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f10a9640 1 -- 172.21.15.32:0/747248135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80abaf0 con 0x7f53ec15cb90 2023-10-16T08:18:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 -- 172.21.15.32:0/747248135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec15cb90 msgr2=0x7f53ec0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 --2- 172.21.15.32:0/747248135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec15cb90 0x7f53ec0cfb10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0725c0 tx=0x7f53e801c440 comp rx=0 tx=0).stop 2023-10-16T08:18:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 -- 172.21.15.32:0/747248135 shutdown_connections 2023-10-16T08:18:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 --2- 172.21.15.32:0/747248135 >> v2:172.21.15.186:3300/0 conn(0x7f53ec0d1400 0x7f53ec0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 --2- 172.21.15.32:0/747248135 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec0d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 --2- 172.21.15.32:0/747248135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec15cb90 0x7f53ec0cfb10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 -- 172.21.15.32:0/747248135 >> 172.21.15.32:0/747248135 conn(0x7f53ec09bc50 msgr2=0x7f53ec0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 -- 172.21.15.32:0/747248135 shutdown_connections 2023-10-16T08:18:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.993+0000 7f53f430f640 1 -- 172.21.15.32:0/747248135 wait complete. 2023-10-16T08:18:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 Processor -- start 2023-10-16T08:18:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 -- start start 2023-10-16T08:18:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 0x7f53ec0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53ec15cb90 0x7f53ec153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec15ba90 con 0x7f53ec0d1400 2023-10-16T08:18:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0c9a70 con 0x7f53ec15cb90 2023-10-16T08:18:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0c9d50 con 0x7f53ec0d3010 2023-10-16T08:18:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 0x7f53ec0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 0x7f53ec0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55488/0 (socket says 172.21.15.32:55488) 2023-10-16T08:18:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 -- 172.21.15.32:0/585577457 learned_addr learned my addr 172.21.15.32:0/585577457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f28ac640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53ec15cb90 0x7f53ec153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f18aa640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 -- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 msgr2=0x7f53ec1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec1538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 -- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53ec15cb90 msgr2=0x7f53ec153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53ec15cb90 0x7f53ec153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 -- 172.21.15.32:0/585577457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e8009480 con 0x7f53ec0d1400 2023-10-16T08:18:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f18aa640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f20ab640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 0x7f53ec0b3d80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f53e801aaf0 tx=0x7f53e80191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80abce0 con 0x7f53ec0d1400 2023-10-16T08:18:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80abe60 con 0x7f53ec0d1400 2023-10-16T08:18:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8003860 con 0x7f53ec0d1400 2023-10-16T08:18:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec0ca030 con 0x7f53ec0d1400 2023-10-16T08:18:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:44.997+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec15b600 con 0x7f53ec0d1400 2023-10-16T08:18:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.001+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec15b7f0 con 0x7f53ec0d1400 2023-10-16T08:18:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.001+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53e80039c0 con 0x7f53ec0d1400 2023-10-16T08:18:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.001+0000 7f53e2ffd640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53d00664a0 0x7f53d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.001+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f53e811e8b0 con 0x7f53ec0d1400 2023-10-16T08:18:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.001+0000 7f53f18aa640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53d00664a0 0x7f53d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.001+0000 7f53f18aa640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53d00664a0 0x7f53d0068960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0b5670 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.005+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f53e80ebee0 con 0x7f53ec0d1400 2023-10-16T08:18:45.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.193+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53ec074010 con 0x7f53ec0d1400 2023-10-16T08:18:45.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.193+0000 7f53e2ffd640 1 -- 172.21.15.32:0/585577457 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f53e80c2020 con 0x7f53ec0d1400 2023-10-16T08:18:45.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:45.200 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/2558825359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2558825359},{"type":"v1","addr":"172.21.15.186:6836","nonce":2558825359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1982601137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1982601137},{"type":"v1","addr":"172.21.15.186:6837","nonce":1982601137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5517,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6833/996452387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":996452387},{"type":"v1","addr":"172.21.15.32:6833","nonce":996452387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":6102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3145231004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3145231004},{"type":"v1","addr":"172.21.15.32:6835","nonce":3145231004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[]} 2023-10-16T08:18:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2023-10-16T08:18:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53d00664a0 msgr2=0x7f53d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53d00664a0 0x7f53d0068960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0b5670 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2023-10-16T08:18:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 msgr2=0x7f53ec0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 0x7f53ec0b3d80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f53e801aaf0 tx=0x7f53e80191e0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 shutdown_connections 2023-10-16T08:18:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53ec15cb90 0x7f53ec153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f53d00664a0 0x7f53d0068960 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53ec0d3010 0x7f53ec1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 --2- 172.21.15.32:0/585577457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0d1400 0x7f53ec0b3d80 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 >> 172.21.15.32:0/585577457 conn(0x7f53ec09bc50 msgr2=0x7f53ec155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:45.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 shutdown_connections 2023-10-16T08:18:45.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.197+0000 7f53f430f640 1 -- 172.21.15.32:0/585577457 wait complete. 2023-10-16T08:18:45.222 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T08:18:45.222 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T08:18:45.223 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T08:18:45.223 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:45.223 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T08:18:45.223 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T08:18:45.224 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T08:18:45.224 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:45.224 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T08:18:45.224 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T08:18:45.225 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T08:18:45.225 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:45.225 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T08:18:45.226 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T08:18:45.226 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T08:18:45.226 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:18:45.235 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T08:18:45.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T08:18:45.237 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T08:18:45.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T08:18:45.240 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T08:18:45.240 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T08:18:45.241 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T08:18:45.241 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T08:18:45.286 INFO:tasks.ceph.mds.d:Started 2023-10-16T08:18:45.286 DEBUG:teuthology.parallel:result is None 2023-10-16T08:18:45.287 INFO:tasks.ceph.mds.a:Started 2023-10-16T08:18:45.287 DEBUG:teuthology.parallel:result is None 2023-10-16T08:18:45.288 INFO:tasks.ceph.mds.b:Started 2023-10-16T08:18:45.289 DEBUG:teuthology.parallel:result is None 2023-10-16T08:18:45.290 INFO:tasks.ceph.mds.c:Started 2023-10-16T08:18:45.290 DEBUG:teuthology.parallel:result is None 2023-10-16T08:18:45.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T08:18:45.376 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-10-16T08:18:45.376 INFO:tasks.ceph.mds.d.smithi186.stdout:starting mds.d at 2023-10-16T08:18:45.376 INFO:tasks.ceph.mds.b.smithi186.stdout:starting mds.b at 2023-10-16T08:18:45.377 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-10-16T08:18:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 Processor -- start 2023-10-16T08:18:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 -- start start 2023-10-16T08:18:45.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88b4070c40 0x7f88b415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b415be60 con 0x7f88b40b0620 2023-10-16T08:18:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b415b3b0 con 0x7f88b4070c40 2023-10-16T08:18:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b94a9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40b0a90 con 0x7f88b40b4020 2023-10-16T08:18:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f88b4070c40 0x7f88b415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88b40b0620 0x7f88b415c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37236/0 (socket says 172.21.15.32:37236) 2023-10-16T08:18:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b37fe640 1 -- 172.21.15.32:0/1922032205 learned_addr learned my addr 172.21.15.32:0/1922032205 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b27fc640 1 --2- 172.21.15.32:0/1922032205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b2ffd640 1 -- 172.21.15.32:0/1922032205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 msgr2=0x7f88b40b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b2ffd640 1 --2- 172.21.15.32:0/1922032205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b2ffd640 1 -- 172.21.15.32:0/1922032205 >> v1:172.21.15.32:6789/0 conn(0x7f88b40b0620 legacy=0x7f88b415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b2ffd640 1 -- 172.21.15.32:0/1922032205 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b415c1f0 con 0x7f88b4070c40 2023-10-16T08:18:45.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.437+0000 7f88b1ffb640 1 -- 172.21.15.32:0/1922032205 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567264564 0 0) 0x7f88b415be60 con 0x7f88b40b0620 2023-10-16T08:18:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b2ffd640 1 --2- 172.21.15.32:0/1922032205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b415ae70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097c0 tx=0x7f88a401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177c2060ce7358c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b1ffb640 1 -- 172.21.15.32:0/1922032205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40ab5f0 con 0x7f88b4070c40 2023-10-16T08:18:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b1ffb640 1 -- 172.21.15.32:0/1922032205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a40ab750 con 0x7f88b4070c40 2023-10-16T08:18:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b1ffb640 1 -- 172.21.15.32:0/1922032205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40abae0 con 0x7f88b4070c40 2023-10-16T08:18:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 -- 172.21.15.32:0/1922032205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 msgr2=0x7f88b415ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1922032205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b415ae70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097c0 tx=0x7f88a401c3f0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 -- 172.21.15.32:0/1922032205 shutdown_connections 2023-10-16T08:18:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1922032205 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1922032205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b415ae70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 -- 172.21.15.32:0/1922032205 >> 172.21.15.32:0/1922032205 conn(0x7f88b406c190 msgr2=0x7f88b406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 -- 172.21.15.32:0/1922032205 shutdown_connections 2023-10-16T08:18:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 -- 172.21.15.32:0/1922032205 wait complete. 2023-10-16T08:18:45.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 Processor -- start 2023-10-16T08:18:45.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.441+0000 7f88b94a9640 1 -- start start 2023-10-16T08:18:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b40b0620 0x7f88b40c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40c4540 con 0x7f88b4070c40 2023-10-16T08:18:45.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4150190 con 0x7f88b40b0620 2023-10-16T08:18:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40c4ea0 con 0x7f88b40b4020 2023-10-16T08:18:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b27fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b40b0620 0x7f88b40c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b27fc640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b40b0620 0x7f88b40c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:38328/0 (socket says 172.21.15.32:38328) 2023-10-16T08:18:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b27fc640 1 -- 172.21.15.32:0/1023852332 learned_addr learned my addr 172.21.15.32:0/1023852332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b37fe640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 -- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 msgr2=0x7f88b40c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 -- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b40b0620 msgr2=0x7f88b40c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b40b0620 0x7f88b40c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 -- 172.21.15.32:0/1023852332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009420 con 0x7f88b4070c40 2023-10-16T08:18:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b37fe640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b2ffd640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b414fc50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f88a40ac000 tx=0x7f88a401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40bf610 con 0x7f88b4070c40 2023-10-16T08:18:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a40bf770 con 0x7f88b4070c40 2023-10-16T08:18:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40bfb20 con 0x7f88b4070c40 2023-10-16T08:18:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b40c38c0 con 0x7f88b4070c40 2023-10-16T08:18:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b40c3e80 con 0x7f88b4070c40 2023-10-16T08:18:45.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.445+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b40c47b0 con 0x7f88b4070c40 2023-10-16T08:18:45.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.449+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f88a40bfc80 con 0x7f88b4070c40 2023-10-16T08:18:45.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.449+0000 7f889b7fe640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f88a00664a0 0x7f88a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.449+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f88a40a8050 con 0x7f88b4070c40 2023-10-16T08:18:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.449+0000 7f88b27fc640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f88a00664a0 0x7f88a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.449+0000 7f88b27fc640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f88a00664a0 0x7f88a0068960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88b41507f0 tx=0x7f88a8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.453+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88a40ebdc0 con 0x7f88b4070c40 2023-10-16T08:18:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:45.621+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f88b4074010 con 0x7f88b4070c40 2023-10-16T08:18:46.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.233+0000 7f889b7fe640 1 -- 172.21.15.32:0/1023852332 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7f88a40ef3e0 con 0x7f88b4070c40 2023-10-16T08:18:46.239 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-10-16T08:18:46.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f88a00664a0 msgr2=0x7f88a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f88a00664a0 0x7f88a0068960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88b41507f0 tx=0x7f88a8009190 comp rx=0 tx=0).stop 2023-10-16T08:18:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 msgr2=0x7f88b414fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b414fc50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f88a40ac000 tx=0x7f88a401ac30 comp rx=0 tx=0).stop 2023-10-16T08:18:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 shutdown_connections 2023-10-16T08:18:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f88a00664a0 0x7f88a0068960 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f88b40b4020 0x7f88b40c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b40b0620 0x7f88b40c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 --2- 172.21.15.32:0/1023852332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88b4070c40 0x7f88b414fc50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 >> 172.21.15.32:0/1023852332 conn(0x7f88b406c190 msgr2=0x7f88b4072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 shutdown_connections 2023-10-16T08:18:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.237+0000 7f88b94a9640 1 -- 172.21.15.32:0/1023852332 wait complete. 2023-10-16T08:18:46.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T08:18:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.409+0000 7f9e9013e640 1 Processor -- start 2023-10-16T08:18:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.409+0000 7f9e9013e640 1 -- start start 2023-10-16T08:18:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e9013e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ca50 0x7f9e8806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e9013e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9e8815ce30 0x7f9e880cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e9013e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e881607e0 con 0x7f9e880b21c0 2023-10-16T08:18:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e9013e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880cf060 con 0x7f9e8815ce30 2023-10-16T08:18:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e9013e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88070210 con 0x7f9e8815ca50 2023-10-16T08:18:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8deda640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ca50 0x7f9e8806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8e6db640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f9e880b21c0 0x7f9e88160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:51442/0 (socket says 172.21.15.32:51442) 2023-10-16T08:18:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8e6db640 1 -- 172.21.15.32:0/885145673 learned_addr learned my addr 172.21.15.32:0/885145673 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8d6d9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9e8815ce30 0x7f9e880cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8ced8640 1 -- 172.21.15.32:0/885145673 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062398688 0 0) 0x7f9e881607e0 con 0x7f9e880b21c0 2023-10-16T08:18:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8ced8640 1 -- 172.21.15.32:0/885145673 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e74003530 con 0x7f9e880b21c0 2023-10-16T08:18:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8deda640 1 -- 172.21.15.32:0/885145673 >> v2:172.21.15.186:3300/0 conn(0x7f9e8815ce30 msgr2=0x7f9e880cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8deda640 1 --2- 172.21.15.32:0/885145673 >> v2:172.21.15.186:3300/0 conn(0x7f9e8815ce30 0x7f9e880cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8deda640 1 -- 172.21.15.32:0/885145673 >> v1:172.21.15.186:6789/0 conn(0x7f9e880b21c0 legacy=0x7f9e88160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.413+0000 7f9e8deda640 1 -- 172.21.15.32:0/885145673 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880704f0 con 0x7f9e8815ca50 2023-10-16T08:18:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e8deda640 1 --2- 172.21.15.32:0/885145673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ca50 0x7f9e8806c980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9e840097c0 tx=0x7f9e8401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae269a3d18188c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e8ced8640 1 -- 172.21.15.32:0/885145673 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e840ad660 con 0x7f9e8815ca50 2023-10-16T08:18:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e8ced8640 1 -- 172.21.15.32:0/885145673 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e840ad7c0 con 0x7f9e8815ca50 2023-10-16T08:18:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e8ced8640 1 -- 172.21.15.32:0/885145673 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e840adb50 con 0x7f9e8815ca50 2023-10-16T08:18:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 -- 172.21.15.32:0/885145673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ca50 msgr2=0x7f9e8806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 --2- 172.21.15.32:0/885145673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ca50 0x7f9e8806c980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9e840097c0 tx=0x7f9e8401c450 comp rx=0 tx=0).stop 2023-10-16T08:18:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 -- 172.21.15.32:0/885145673 shutdown_connections 2023-10-16T08:18:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 --2- 172.21.15.32:0/885145673 >> v2:172.21.15.186:3300/0 conn(0x7f9e8815ce30 0x7f9e880cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 --2- 172.21.15.32:0/885145673 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ca50 0x7f9e8806c980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 -- 172.21.15.32:0/885145673 >> 172.21.15.32:0/885145673 conn(0x7f9e880cd1c0 msgr2=0x7f9e880cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 -- 172.21.15.32:0/885145673 shutdown_connections 2023-10-16T08:18:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 -- 172.21.15.32:0/885145673 wait complete. 2023-10-16T08:18:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 Processor -- start 2023-10-16T08:18:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.417+0000 7f9e9013e640 1 -- start start 2023-10-16T08:18:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 0x7f9e88157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8815ca50 0x7f9e881f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ce30 0x7f9e880b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880c4d90 con 0x7f9e880b21c0 2023-10-16T08:18:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880c43f0 con 0x7f9e8815ca50 2023-10-16T08:18:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880c46d0 con 0x7f9e8815ce30 2023-10-16T08:18:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 0x7f9e88157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 0x7f9e88157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55548/0 (socket says 172.21.15.32:55548) 2023-10-16T08:18:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 -- 172.21.15.32:0/220384857 learned_addr learned my addr 172.21.15.32:0/220384857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 -- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ce30 msgr2=0x7f9e880b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ce30 0x7f9e880b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 -- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8815ca50 msgr2=0x7f9e881f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8815ca50 0x7f9e881f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 -- 172.21.15.32:0/220384857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e84009420 con 0x7f9e880b21c0 2023-10-16T08:18:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e8deda640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 0x7f9e88157fd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9e840b0040 tx=0x7f9e840191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e840b6610 con 0x7f9e880b21c0 2023-10-16T08:18:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e840b6770 con 0x7f9e880b21c0 2023-10-16T08:18:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e880b2b60 con 0x7f9e880b21c0 2023-10-16T08:18:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e840b6b20 con 0x7f9e880b21c0 2023-10-16T08:18:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e880c66a0 con 0x7f9e880b21c0 2023-10-16T08:18:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e88074010 con 0x7f9e880b21c0 2023-10-16T08:18:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.421+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e840b4040 con 0x7f9e880b21c0 2023-10-16T08:18:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.425+0000 7f9e7e7fc640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e6c0664a0 0x7f9e6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.425+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9e8411df40 con 0x7f9e880b21c0 2023-10-16T08:18:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.425+0000 7f9e8d6d9640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e6c0664a0 0x7f9e6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.425+0000 7f9e8d6d9640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e6c0664a0 0x7f9e6c068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006aa0 tx=0x7f9e78006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.429+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e840eb660 con 0x7f9e880b21c0 2023-10-16T08:18:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9e880cddb0 con 0x7f9e880b21c0 2023-10-16T08:18:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e7e7fc640 1 -- 172.21.15.32:0/220384857 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5303 (secure 0 0 0) 0x7f9e840b2090 con 0x7f9e880b21c0 2023-10-16T08:18:46.643 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:46.644 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:18:46.644 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-10-16T08:18:46.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.644 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-10-16T08:18:46.644 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8Sxlq7jUBRAAZ/PoSPxeTxzGVyZoXvddGA==", 2023-10-16T08:18:46.645 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.645 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.645 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:18:46.645 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:18:46.645 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.646 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-10-16T08:18:46.646 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlD5VrEhAAoIJTZvkt2JVMQDjd9QNNnA==", 2023-10-16T08:18:46.647 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.647 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.647 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:18:46.647 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:18:46.647 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.648 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-10-16T08:18:46.648 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlZ8QwDRAAa6bkfiFmL7N90yyiFUBOsg==", 2023-10-16T08:18:46.649 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:18:46.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:18:46.649 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.650 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-10-16T08:18:46.650 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxliSn5GRAA8atnMBpEoXUyjpqQtoGWgw==", 2023-10-16T08:18:46.651 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:18:46.651 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:18:46.651 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.652 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-10-16T08:18:46.652 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAZ8SxltyuHBRAA523N7XKVdNMwSLQo24Vvvg==", 2023-10-16T08:18:46.652 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.654 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-10-16T08:18:46.654 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAc8Sxl88vjCxAAh5b4FThP+7Y7k0nYLylx5Q==", 2023-10-16T08:18:46.654 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.654 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.655 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.655 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.656 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-10-16T08:18:46.656 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAf8SxlfcEoEhAABJMIQKe43NGUqMCXCq6HEA==", 2023-10-16T08:18:46.656 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.656 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.656 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-10-16T08:18:46.657 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAi8SxlomRcGRAAIu96ZzCy18pSgsR3hD/Y7Q==", 2023-10-16T08:18:46.658 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.658 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.658 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.658 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.659 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-10-16T08:18:46.659 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAo8SxlXxEEHxAAqohCELe6In6sSjbBTHY+OQ==", 2023-10-16T08:18:46.659 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.660 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.660 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.660 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.661 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-10-16T08:18:46.661 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAr8SxloVNhJhAADF2erI7/queuEM5kSqqJkQ==", 2023-10-16T08:18:46.661 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.661 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.661 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.662 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.662 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-10-16T08:18:46.663 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAu8SxlRtStLhAAIzAuqOD9H7iiabi1P67nNA==", 2023-10-16T08:18:46.663 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.663 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.663 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.663 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.664 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.664 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-10-16T08:18:46.664 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAx8SxlthuoNRAAubSeLAllgyLqaRY4z4G35w==", 2023-10-16T08:18:46.665 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.665 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-10-16T08:18:46.665 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-10-16T08:18:46.665 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.666 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-10-16T08:18:46.666 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ==", 2023-10-16T08:18:46.666 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.667 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.667 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-10-16T08:18:46.667 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw pool=testcephfs1_data" 2023-10-16T08:18:46.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.668 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-10-16T08:18:46.668 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlA8EAKBAA7lsMEVeNC7t3xvgCLWoYcQ==", 2023-10-16T08:18:46.668 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.668 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.669 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:18:46.669 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:18:46.669 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:18:46.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.670 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-10-16T08:18:46.670 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlBylLJBAAXNMyfzVb5MDsSdSxVN59lg==", 2023-10-16T08:18:46.670 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.670 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.671 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:18:46.671 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:18:46.671 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:18:46.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.672 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-10-16T08:18:46.672 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8Sxl1dxZLhAAqhBqKEmTcorRaKo5YmmzvA==", 2023-10-16T08:18:46.672 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.672 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-10-16T08:18:46.673 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-10-16T08:18:46.673 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-10-16T08:18:46.673 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-10-16T08:18:46.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.674 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-10-16T08:18:46.674 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAW8SxljlvpIxAA4Re4iMDl0YMQvX5A8wqXaQ==", 2023-10-16T08:18:46.674 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.674 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-10-16T08:18:46.674 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-10-16T08:18:46.675 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-10-16T08:18:46.675 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.676 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-10-16T08:18:46.676 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAX8SxlT7Q7ARAAVxrt9krDiqnEkQ0+I4duzQ==", 2023-10-16T08:18:46.676 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.676 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-10-16T08:18:46.676 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-10-16T08:18:46.677 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:46.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:46.678 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-10-16T08:18:46.678 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAW8SxlPtfFOBAAOXlHIg/7e75c4MipG5PHGw==", 2023-10-16T08:18:46.678 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-10-16T08:18:46.678 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-10-16T08:18:46.678 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-10-16T08:18:46.679 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-10-16T08:18:46.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:46.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-10-16T08:18:46.679 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:18:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e6c0664a0 msgr2=0x7f9e6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e6c0664a0 0x7f9e6c068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006aa0 tx=0x7f9e78006a30 comp rx=0 tx=0).stop 2023-10-16T08:18:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 msgr2=0x7f9e88157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 0x7f9e88157fd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9e840b0040 tx=0x7f9e840191f0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 shutdown_connections 2023-10-16T08:18:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9e6c0664a0 0x7f9e6c068960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9e8815ce30 0x7f9e880b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e8815ca50 0x7f9e881f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 --2- 172.21.15.32:0/220384857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e880b21c0 0x7f9e88157fd0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 >> 172.21.15.32:0/220384857 conn(0x7f9e880cd1c0 msgr2=0x7f9e8815c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 shutdown_connections 2023-10-16T08:18:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.637+0000 7f9e9013e640 1 -- 172.21.15.32:0/220384857 wait complete. 2023-10-16T08:18:46.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:46.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 Processor -- start 2023-10-16T08:18:46.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 -- start start 2023-10-16T08:18:46.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f147815cb90 0x7f14780cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f14780d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780ce770 con 0x7f14780d1400 2023-10-16T08:18:46.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14780d1890 con 0x7f147815cb90 2023-10-16T08:18:46.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f147d69e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780d0600 con 0x7f14780d3010 2023-10-16T08:18:46.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f14780d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f14780d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:51114/0 (socket says 172.21.15.32:51114) 2023-10-16T08:18:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 -- 172.21.15.32:0/3818083766 learned_addr learned my addr 172.21.15.32:0/3818083766 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f1476ffd640 1 --2- 172.21.15.32:0/3818083766 >> v2:172.21.15.186:3300/0 conn(0x7f147815cb90 0x7f14780cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f1475ffb640 1 -- 172.21.15.32:0/3818083766 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96043688 0 0) 0x7f14780ce770 con 0x7f14780d1400 2023-10-16T08:18:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f1475ffb640 1 -- 172.21.15.32:0/3818083766 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1460003530 con 0x7f14780d1400 2023-10-16T08:18:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 -- 172.21.15.32:0/3818083766 >> v2:172.21.15.186:3300/0 conn(0x7f147815cb90 msgr2=0x7f14780cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 --2- 172.21.15.32:0/3818083766 >> v2:172.21.15.186:3300/0 conn(0x7f147815cb90 0x7f14780cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 -- 172.21.15.32:0/3818083766 >> v1:172.21.15.186:6789/0 conn(0x7f14780d1400 legacy=0x7f14780cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f14767fc640 1 -- 172.21.15.32:0/3818083766 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780c6450 con 0x7f14780d3010 2023-10-16T08:18:46.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.821+0000 7f1476ffd640 1 --2- 172.21.15.32:0/3818083766 >> v2:172.21.15.186:3300/0 conn(0x7f147815cb90 0x7f14780cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.825+0000 7f14767fc640 1 --2- 172.21.15.32:0/3818083766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f14780d0ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f146c009870 tx=0x7f146c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a77b9a5eab8f31 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.825+0000 7f1475ffb640 1 -- 172.21.15.32:0/3818083766 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c0ab680 con 0x7f14780d3010 2023-10-16T08:18:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.825+0000 7f1475ffb640 1 -- 172.21.15.32:0/3818083766 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c0ab7e0 con 0x7f14780d3010 2023-10-16T08:18:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.825+0000 7f1475ffb640 1 -- 172.21.15.32:0/3818083766 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c0abb70 con 0x7f14780d3010 2023-10-16T08:18:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.825+0000 7f147d69e640 1 -- 172.21.15.32:0/3818083766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 msgr2=0x7f14780d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.825+0000 7f147d69e640 1 --2- 172.21.15.32:0/3818083766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f14780d0ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f146c009870 tx=0x7f146c01c480 comp rx=0 tx=0).stop 2023-10-16T08:18:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- 172.21.15.32:0/3818083766 shutdown_connections 2023-10-16T08:18:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 --2- 172.21.15.32:0/3818083766 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f14780d0ec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 --2- 172.21.15.32:0/3818083766 >> v2:172.21.15.186:3300/0 conn(0x7f147815cb90 0x7f14780cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- 172.21.15.32:0/3818083766 >> 172.21.15.32:0/3818083766 conn(0x7f147809bc50 msgr2=0x7f14780d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- 172.21.15.32:0/3818083766 shutdown_connections 2023-10-16T08:18:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- 172.21.15.32:0/3818083766 wait complete. 2023-10-16T08:18:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 Processor -- start 2023-10-16T08:18:46.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- start start 2023-10-16T08:18:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14780d1400 0x7f1478152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f147815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 0x7f147815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478154cb0 con 0x7f147815cb90 2023-10-16T08:18:46.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781540d0 con 0x7f14780d1400 2023-10-16T08:18:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f147d69e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14781534c0 con 0x7f14780d3010 2023-10-16T08:18:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 0x7f147815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 0x7f147815b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55556/0 (socket says 172.21.15.32:55556) 2023-10-16T08:18:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 -- 172.21.15.32:0/4100061437 learned_addr learned my addr 172.21.15.32:0/4100061437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f1476ffd640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14780d1400 0x7f1478152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14767fc640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f147815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 -- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 msgr2=0x7f147815ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f147815ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 -- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14780d1400 msgr2=0x7f1478152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:46.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14780d1400 0x7f1478152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 -- 172.21.15.32:0/4100061437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680093b0 con 0x7f147815cb90 2023-10-16T08:18:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f1476ffd640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14780d1400 0x7f1478152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.829+0000 7f14777fe640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 0x7f147815b350 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f147000c730 tx=0x7f147000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470003b80 con 0x7f147815cb90 2023-10-16T08:18:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470003ce0 con 0x7f147815cb90 2023-10-16T08:18:46.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000aa90 con 0x7f147815cb90 2023-10-16T08:18:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0094d0 con 0x7f147815cb90 2023-10-16T08:18:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1478153a80 con 0x7f147815cb90 2023-10-16T08:18:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478154260 con 0x7f147815cb90 2023-10-16T08:18:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f147000abf0 con 0x7f147815cb90 2023-10-16T08:18:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f145f7fe640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14640664a0 0x7f1464068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1470083a40 con 0x7f147815cb90 2023-10-16T08:18:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.833+0000 7f1476ffd640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14640664a0 0x7f1464068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:46.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.837+0000 7f1476ffd640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14640664a0 0x7f1464068960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1468009d20 tx=0x7f1468008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:46.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.837+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14700510c0 con 0x7f147815cb90 2023-10-16T08:18:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1478074010 con 0x7f147815cb90 2023-10-16T08:18:47.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f145f7fe640 1 -- 172.21.15.32:0/4100061437 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13062 (secure 0 0 0) 0x7f14700546e0 con 0x7f147815cb90 2023-10-16T08:18:47.003 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:46.251024+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14640664a0 msgr2=0x7f1464068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14640664a0 0x7f1464068960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1468009d20 tx=0x7f1468008040 comp rx=0 tx=0).stop 2023-10-16T08:18:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 msgr2=0x7f147815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 0x7f147815b350 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f147000c730 tx=0x7f147000cbf0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 shutdown_connections 2023-10-16T08:18:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f147815cb90 0x7f147815b350 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f14640664a0 0x7f1464068960 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f14780d3010 0x7f147815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 --2- 172.21.15.32:0/4100061437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14780d1400 0x7f1478152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:46.997+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 >> 172.21.15.32:0/4100061437 conn(0x7f147809bc50 msgr2=0x7f14780b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.001+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 shutdown_connections 2023-10-16T08:18:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.001+0000 7f147d69e640 1 -- 172.21.15.32:0/4100061437 wait complete. 2023-10-16T08:18:47.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:47.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 Processor -- start 2023-10-16T08:18:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 -- start start 2023-10-16T08:18:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f8390069e90 0x7f839006d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f839006a270 0x7f83900c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839015ae30 con 0x7f83900702f0 2023-10-16T08:18:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900cb090 con 0x7f839006a270 2023-10-16T08:18:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f839666b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900ca510 con 0x7f8390069e90 2023-10-16T08:18:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f839006a270 0x7f83900c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f8394c08640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83900702f0 0x7f8390070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37244/0 (socket says 172.21.15.32:37244) 2023-10-16T08:18:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f8394c08640 1 -- 172.21.15.32:0/12442089 learned_addr learned my addr 172.21.15.32:0/12442089 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838ffff640 1 --2- 172.21.15.32:0/12442089 >> v2:172.21.15.186:3300/0 conn(0x7f8390069e90 0x7f839006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838f7fe640 1 -- 172.21.15.32:0/12442089 >> v2:172.21.15.186:3300/0 conn(0x7f8390069e90 msgr2=0x7f839006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838f7fe640 1 --2- 172.21.15.32:0/12442089 >> v2:172.21.15.186:3300/0 conn(0x7f8390069e90 0x7f839006d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838f7fe640 1 -- 172.21.15.32:0/12442089 >> v1:172.21.15.32:6789/0 conn(0x7f83900702f0 legacy=0x7f8390070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838f7fe640 1 -- 172.21.15.32:0/12442089 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900ca7f0 con 0x7f839006a270 2023-10-16T08:18:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.169+0000 7f838effd640 1 -- 172.21.15.32:0/12442089 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721628071 0 0) 0x7f839015ae30 con 0x7f83900702f0 2023-10-16T08:18:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f838f7fe640 1 --2- 172.21.15.32:0/12442089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839006a270 0x7f83900c9ce0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8390071590 tx=0x7f838401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4466d19ae2fbf2eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f838effd640 1 -- 172.21.15.32:0/12442089 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840a7070 con 0x7f839006a270 2023-10-16T08:18:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f838effd640 1 -- 172.21.15.32:0/12442089 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83840ab430 con 0x7f839006a270 2023-10-16T08:18:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f838effd640 1 -- 172.21.15.32:0/12442089 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840ab7c0 con 0x7f839006a270 2023-10-16T08:18:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 -- 172.21.15.32:0/12442089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839006a270 msgr2=0x7f83900c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 --2- 172.21.15.32:0/12442089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839006a270 0x7f83900c9ce0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8390071590 tx=0x7f838401c430 comp rx=0 tx=0).stop 2023-10-16T08:18:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 -- 172.21.15.32:0/12442089 shutdown_connections 2023-10-16T08:18:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 --2- 172.21.15.32:0/12442089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f839006a270 0x7f83900c9ce0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 --2- 172.21.15.32:0/12442089 >> v2:172.21.15.186:3300/0 conn(0x7f8390069e90 0x7f839006d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 -- 172.21.15.32:0/12442089 >> 172.21.15.32:0/12442089 conn(0x7f83900b0da0 msgr2=0x7f83900ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 -- 172.21.15.32:0/12442089 shutdown_connections 2023-10-16T08:18:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 -- 172.21.15.32:0/12442089 wait complete. 2023-10-16T08:18:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 Processor -- start 2023-10-16T08:18:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.173+0000 7f839666b640 1 -- start start 2023-10-16T08:18:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8390069e90 0x7f83900c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839006a270 0x7f83900c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 0x7f83900cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900c74e0 con 0x7f83900702f0 2023-10-16T08:18:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901f7820 con 0x7f839006a270 2023-10-16T08:18:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900c9570 con 0x7f8390069e90 2023-10-16T08:18:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 0x7f83900cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 0x7f83900cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55570/0 (socket says 172.21.15.32:55570) 2023-10-16T08:18:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 -- 172.21.15.32:0/1958545796 learned_addr learned my addr 172.21.15.32:0/1958545796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f838f7fe640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839006a270 0x7f83900c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f838ffff640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8390069e90 0x7f83900c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 -- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8390069e90 msgr2=0x7f83900c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8390069e90 0x7f83900c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 -- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839006a270 msgr2=0x7f83900c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839006a270 0x7f83900c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 -- 172.21.15.32:0/1958545796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384009420 con 0x7f83900702f0 2023-10-16T08:18:47.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f838ffff640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8390069e90 0x7f83900c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f8394c08640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 0x7f83900cb1d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f838800c780 tx=0x7f838800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838800ce90 con 0x7f83900702f0 2023-10-16T08:18:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838800a640 con 0x7f83900702f0 2023-10-16T08:18:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838800a9d0 con 0x7f83900702f0 2023-10-16T08:18:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83900c9f40 con 0x7f83900702f0 2023-10-16T08:18:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83900c8f20 con 0x7f83900702f0 2023-10-16T08:18:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.177+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83900c9700 con 0x7f83900702f0 2023-10-16T08:18:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.181+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f838800ab30 con 0x7f83900702f0 2023-10-16T08:18:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.181+0000 7f838cff9640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f837c0664d0 0x7f837c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.181+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f83880830a0 con 0x7f83900702f0 2023-10-16T08:18:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.181+0000 7f838ffff640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f837c0664d0 0x7f837c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.181+0000 7f838ffff640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f837c0664d0 0x7f837c068990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8380007a70 tx=0x7f8380008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.185+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8388050740 con 0x7f83900702f0 2023-10-16T08:18:47.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:18:47.337 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:18:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.345+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f839004e170 con 0x7f83900702f0 2023-10-16T08:18:47.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.345+0000 7f838cff9640 1 -- 172.21.15.32:0/1958545796 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8388053d60 con 0x7f83900702f0 2023-10-16T08:18:47.352 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f837c0664d0 msgr2=0x7f837c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f837c0664d0 0x7f837c068990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8380007a70 tx=0x7f8380008040 comp rx=0 tx=0).stop 2023-10-16T08:18:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 msgr2=0x7f83900cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 0x7f83900cb1d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f838800c780 tx=0x7f838800cc40 comp rx=0 tx=0).stop 2023-10-16T08:18:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 shutdown_connections 2023-10-16T08:18:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83900702f0 0x7f83900cb1d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839006a270 0x7f83900c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f837c0664d0 0x7f837c068990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 --2- 172.21.15.32:0/1958545796 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8390069e90 0x7f83900c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 >> 172.21.15.32:0/1958545796 conn(0x7f83900b0da0 msgr2=0x7f83900ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 shutdown_connections 2023-10-16T08:18:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.349+0000 7f839666b640 1 -- 172.21.15.32:0/1958545796 wait complete. 2023-10-16T08:18:47.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T08:18:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 Processor -- start 2023-10-16T08:18:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 -- start start 2023-10-16T08:18:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37ec0cb4c0 0x7f37ec0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec15bad0 con 0x7f37ec0cee80 2023-10-16T08:18:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec15c090 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f2dc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec15bdb0 con 0x7f37ec15a7d0 2023-10-16T08:18:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f1362640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37ec0cb4c0 0x7f37ec0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37ebfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37ec0cee80 0x7f37ec0cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37248/0 (socket says 172.21.15.32:37248) 2023-10-16T08:18:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37ebfff640 1 -- 172.21.15.32:0/43164450 learned_addr learned my addr 172.21.15.32:0/43164450 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.569+0000 7f37f0b61640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f1362640 1 -- 172.21.15.32:0/43164450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 msgr2=0x7f37ec0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f1362640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec0cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f1362640 1 -- 172.21.15.32:0/43164450 >> v1:172.21.15.32:6789/0 conn(0x7f37ec0cee80 legacy=0x7f37ec0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f1362640 1 -- 172.21.15.32:0/43164450 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0cd380 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f0b61640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37eb7fe640 1 -- 172.21.15.32:0/43164450 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897123063 0 0) 0x7f37ec15bad0 con 0x7f37ec0cee80 2023-10-16T08:18:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f1362640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec0cb950 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009d90 tx=0x7f37e401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f1b2f57271b193 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37eb7fe640 1 -- 172.21.15.32:0/43164450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e40a5a00 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37eb7fe640 1 -- 172.21.15.32:0/43164450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e40a5b60 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37eb7fe640 1 -- 172.21.15.32:0/43164450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e40ad440 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 -- 172.21.15.32:0/43164450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 msgr2=0x7f37ec0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec0cb950 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009d90 tx=0x7f37e401c920 comp rx=0 tx=0).stop 2023-10-16T08:18:47.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 -- 172.21.15.32:0/43164450 shutdown_connections 2023-10-16T08:18:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec0cb950 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/43164450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 -- 172.21.15.32:0/43164450 >> 172.21.15.32:0/43164450 conn(0x7f37ec06a8c0 msgr2=0x7f37ec099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.573+0000 7f37f2dc5640 1 -- 172.21.15.32:0/43164450 shutdown_connections 2023-10-16T08:18:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 -- 172.21.15.32:0/43164450 wait complete. 2023-10-16T08:18:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 Processor -- start 2023-10-16T08:18:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 -- start start 2023-10-16T08:18:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec0cee80 0x7f37ec151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec164c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1533e0 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec151f50 con 0x7f37ec0cee80 2023-10-16T08:18:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f2dc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1660c0 con 0x7f37ec15a7d0 2023-10-16T08:18:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55594/0 (socket says 172.21.15.32:55594) 2023-10-16T08:18:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 -- 172.21.15.32:0/749844395 learned_addr learned my addr 172.21.15.32:0/749844395 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f1362640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37ebfff640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec0cee80 0x7f37ec151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 -- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 msgr2=0x7f37ec164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec164c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 -- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec0cee80 msgr2=0x7f37ec151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec0cee80 0x7f37ec151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 -- 172.21.15.32:0/749844395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4009a70 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f1362640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37f0b61640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec152ea0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f37e000d800 tx=0x7f37e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00059a0 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e0005b00 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.577+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0005e90 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec1654b0 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec165a70 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec04e170 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f37e0004030 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37e97fa640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f37d00664a0 0x7f37d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f37e0083c20 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37ebfff640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f37d00664a0 0x7f37d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.581+0000 7f37ebfff640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f37d00664a0 0x7f37d0068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f37ec144700 tx=0x7f37d8007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.585+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f37e00512a0 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.745+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f37ec158c70 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.745+0000 7f37e97fa640 1 -- 172.21.15.32:0/749844395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+65 (secure 0 0 0) 0x7f37e00548c0 con 0x7f37ec0cb4c0 2023-10-16T08:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-10-16T08:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-10-16T08:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-10-16T08:18:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-10-16T08:18:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:47.753 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-10-16T08:18:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.745+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f37d00664a0 msgr2=0x7f37d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.745+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f37d00664a0 0x7f37d0068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f37ec144700 tx=0x7f37d8007b60 comp rx=0 tx=0).stop 2023-10-16T08:18:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.745+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 msgr2=0x7f37ec152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.745+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec152ea0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f37e000d800 tx=0x7f37e000dcc0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 shutdown_connections 2023-10-16T08:18:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f37d00664a0 0x7f37d0068960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f37ec15a7d0 0x7f37ec164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37ec0cee80 0x7f37ec151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 --2- 172.21.15.32:0/749844395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37ec0cb4c0 0x7f37ec152ea0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 >> 172.21.15.32:0/749844395 conn(0x7f37ec06a8c0 msgr2=0x7f37ec06f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:47.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 shutdown_connections 2023-10-16T08:18:47.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.749+0000 7f37f2dc5640 1 -- 172.21.15.32:0/749844395 wait complete. 2023-10-16T08:18:47.774 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'testcephfs1' 2023-10-16T08:18:47.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_metadata --pg_num_min 64 2023-10-16T08:18:47.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 Processor -- start 2023-10-16T08:18:47.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 -- start start 2023-10-16T08:18:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd53c0d2570 0x7fd53c161210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd53c0cfd20 0x7fd53c0d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53c0ce460 con 0x7fd53c0cdc80 2023-10-16T08:18:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53c0cd830 con 0x7fd53c0cfd20 2023-10-16T08:18:47.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd541572640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53c15da30 con 0x7fd53c0d2570 2023-10-16T08:18:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd533fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd53c0cdc80 0x7fd53c0cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37254/0 (socket says 172.21.15.32:37254) 2023-10-16T08:18:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd533fff640 1 -- 172.21.15.32:0/3837307649 learned_addr learned my addr 172.21.15.32:0/3837307649 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53b7fe640 1 --2- 172.21.15.32:0/3837307649 >> v2:172.21.15.32:3300/0 conn(0x7fd53c0cfd20 0x7fd53c0d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53affd640 1 --2- 172.21.15.32:0/3837307649 >> v2:172.21.15.186:3300/0 conn(0x7fd53c0d2570 0x7fd53c161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53a7fc640 1 -- 172.21.15.32:0/3837307649 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974187288 0 0) 0x7fd53c0ce460 con 0x7fd53c0cdc80 2023-10-16T08:18:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53a7fc640 1 -- 172.21.15.32:0/3837307649 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd528003530 con 0x7fd53c0cdc80 2023-10-16T08:18:47.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53b7fe640 1 -- 172.21.15.32:0/3837307649 >> v2:172.21.15.186:3300/0 conn(0x7fd53c0d2570 msgr2=0x7fd53c161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53b7fe640 1 --2- 172.21.15.32:0/3837307649 >> v2:172.21.15.186:3300/0 conn(0x7fd53c0d2570 0x7fd53c161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53b7fe640 1 -- 172.21.15.32:0/3837307649 >> v1:172.21.15.32:6789/0 conn(0x7fd53c0cdc80 legacy=0x7fd53c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53b7fe640 1 -- 172.21.15.32:0/3837307649 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c161750 con 0x7fd53c0cfd20 2023-10-16T08:18:47.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.917+0000 7fd53affd640 1 --2- 172.21.15.32:0/3837307649 >> v2:172.21.15.186:3300/0 conn(0x7fd53c0d2570 0x7fd53c161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:47.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd53b7fe640 1 --2- 172.21.15.32:0/3837307649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c0d0e60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0722b0 tx=0x7fd53401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7079d5335b52b701 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd53a7fc640 1 -- 172.21.15.32:0/3837307649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340a58b0 con 0x7fd53c0cfd20 2023-10-16T08:18:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd53a7fc640 1 -- 172.21.15.32:0/3837307649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5340a5a10 con 0x7fd53c0cfd20 2023-10-16T08:18:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd53a7fc640 1 -- 172.21.15.32:0/3837307649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340a5da0 con 0x7fd53c0cfd20 2023-10-16T08:18:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 -- 172.21.15.32:0/3837307649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 msgr2=0x7fd53c0d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 --2- 172.21.15.32:0/3837307649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c0d0e60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0722b0 tx=0x7fd53401c8e0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 -- 172.21.15.32:0/3837307649 shutdown_connections 2023-10-16T08:18:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 --2- 172.21.15.32:0/3837307649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c0d0e60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 --2- 172.21.15.32:0/3837307649 >> v2:172.21.15.186:3300/0 conn(0x7fd53c0d2570 0x7fd53c161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 -- 172.21.15.32:0/3837307649 >> 172.21.15.32:0/3837307649 conn(0x7fd53c0715c0 msgr2=0x7fd53c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 -- 172.21.15.32:0/3837307649 shutdown_connections 2023-10-16T08:18:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 -- 172.21.15.32:0/3837307649 wait complete. 2023-10-16T08:18:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 Processor -- start 2023-10-16T08:18:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.921+0000 7fd541572640 1 -- start start 2023-10-16T08:18:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd53c0cdc80 0x7fd53c150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd53c0d2570 0x7fd53c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c0ccc50 con 0x7fd53c0cfd20 2023-10-16T08:18:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c151a80 con 0x7fd53c0cdc80 2023-10-16T08:18:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c1531d0 con 0x7fd53c0d2570 2023-10-16T08:18:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55624/0 (socket says 172.21.15.32:55624) 2023-10-16T08:18:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 -- 172.21.15.32:0/4188620771 learned_addr learned my addr 172.21.15.32:0/4188620771 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd53affd640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd53c0cdc80 0x7fd53c150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd53b7fe640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd53c0d2570 0x7fd53c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 -- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd53c0d2570 msgr2=0x7fd53c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd53c0d2570 0x7fd53c1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 -- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd53c0cdc80 msgr2=0x7fd53c150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd53c0cdc80 0x7fd53c150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 -- 172.21.15.32:0/4188620771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534009a70 con 0x7fd53c0cfd20 2023-10-16T08:18:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd53b7fe640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd53c0d2570 0x7fd53c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd533fff640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c157f00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c450 tx=0x7fd52400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524005cf0 con 0x7fd53c0cfd20 2023-10-16T08:18:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524005e50 con 0x7fd53c0cfd20 2023-10-16T08:18:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c0ccea0 con 0x7fd53c0cfd20 2023-10-16T08:18:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524007270 con 0x7fd53c0cfd20 2023-10-16T08:18:47.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c151710 con 0x7fd53c0cfd20 2023-10-16T08:18:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c151570 con 0x7fd53c0cfd20 2023-10-16T08:18:47.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.925+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd5240073d0 con 0x7fd53c0cfd20 2023-10-16T08:18:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.929+0000 7fd5337fe640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd51c0664a0 0x7fd51c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.929+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd524083060 con 0x7fd53c0cfd20 2023-10-16T08:18:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.929+0000 7fd53affd640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd51c0664a0 0x7fd51c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:47.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.929+0000 7fd53affd640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd51c0664a0 0x7fd51c068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd53c159e70 tx=0x7fd52c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:47.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:47.933+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd53c151570 con 0x7fd53c0cfd20 2023-10-16T08:18:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.089+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd53c151570 con 0x7fd53c0cfd20 2023-10-16T08:18:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.233+0000 7fd5337fe640 1 -- 172.21.15.32:0/4188620771 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64}]=0 pool 'testcephfs1_metadata' created v51) v1 ==== 148+0+0 (secure 0 0 0) 0x7fd53c151570 con 0x7fd53c0cfd20 2023-10-16T08:18:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_metadata' created 2023-10-16T08:18:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd51c0664a0 msgr2=0x7fd51c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd51c0664a0 0x7fd51c068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd53c159e70 tx=0x7fd52c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 msgr2=0x7fd53c157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c157f00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c450 tx=0x7fd52400c910 comp rx=0 tx=0).stop 2023-10-16T08:18:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 shutdown_connections 2023-10-16T08:18:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd51c0664a0 0x7fd51c068960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd53c0d2570 0x7fd53c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd53c0cfd20 0x7fd53c157f00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 --2- 172.21.15.32:0/4188620771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd53c0cdc80 0x7fd53c150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 >> 172.21.15.32:0/4188620771 conn(0x7fd53c0715c0 msgr2=0x7fd53c15c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 shutdown_connections 2023-10-16T08:18:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.237+0000 7fd541572640 1 -- 172.21.15.32:0/4188620771 wait complete. 2023-10-16T08:18:48.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T08:18:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633c3aa640 1 Processor -- start 2023-10-16T08:18:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633c3aa640 1 -- start start 2023-10-16T08:18:48.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633c3aa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340c3d80 0x7f63340c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:48.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633c3aa640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340addc0 con 0x7f63340ca3c0 2023-10-16T08:18:48.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633c3aa640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340c35c0 con 0x7f63340cd350 2023-10-16T08:18:48.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633c3aa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63340c4750 con 0x7f63340c3d80 2023-10-16T08:18:48.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339945640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f63340ca3c0 0x7f63340cc2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37264/0 (socket says 172.21.15.32:37264) 2023-10-16T08:18:48.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339945640 1 -- 172.21.15.32:0/4057821767 learned_addr learned my addr 172.21.15.32:0/4057821767 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:48.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633a947640 1 --2- 172.21.15.32:0/4057821767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340c3d80 0x7f63340c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:48.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78271983 0 0) 0x7f63340addc0 con 0x7f63340ca3c0 2023-10-16T08:18:48.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6320003530 con 0x7f63340ca3c0 2023-10-16T08:18:48.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521168028 0 0) 0x7f63340c35c0 con 0x7f63340cd350 2023-10-16T08:18:48.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63340addc0 con 0x7f63340cd350 2023-10-16T08:18:48.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633a947640 1 -- 172.21.15.32:0/4057821767 >> v1:172.21.15.186:6789/0 conn(0x7f63340cd350 legacy=0x7f63340c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633a947640 1 -- 172.21.15.32:0/4057821767 >> v1:172.21.15.32:6789/0 conn(0x7f63340ca3c0 legacy=0x7f63340cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215543609 0 0) 0x7f6320003530 con 0x7f63340ca3c0 2023-10-16T08:18:48.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.413+0000 7f633a947640 1 -- 172.21.15.32:0/4057821767 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340ca840 con 0x7f63340c3d80 2023-10-16T08:18:48.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633a947640 1 --2- 172.21.15.32:0/4057821767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340c3d80 0x7f63340c4210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f632c009dc0 tx=0x7f632c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af78e8f5983f77a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:48.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324001680 con 0x7f63340c3d80 2023-10-16T08:18:48.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c0a55c0 con 0x7f63340c3d80 2023-10-16T08:18:48.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f6339144640 1 -- 172.21.15.32:0/4057821767 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0a5950 con 0x7f63340c3d80 2023-10-16T08:18:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 -- 172.21.15.32:0/4057821767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340c3d80 msgr2=0x7f63340c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 --2- 172.21.15.32:0/4057821767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340c3d80 0x7f63340c4210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f632c009dc0 tx=0x7f632c01c890 comp rx=0 tx=0).stop 2023-10-16T08:18:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 -- 172.21.15.32:0/4057821767 shutdown_connections 2023-10-16T08:18:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 --2- 172.21.15.32:0/4057821767 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340c3d80 0x7f63340c4210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 -- 172.21.15.32:0/4057821767 >> 172.21.15.32:0/4057821767 conn(0x7f63340b0d10 msgr2=0x7f63340ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 -- 172.21.15.32:0/4057821767 shutdown_connections 2023-10-16T08:18:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.417+0000 7f633c3aa640 1 -- 172.21.15.32:0/4057821767 wait complete. 2023-10-16T08:18:48.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 Processor -- start 2023-10-16T08:18:48.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 -- start start 2023-10-16T08:18:48.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 0x7f6334154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:48.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340ca3c0 0x7f63340b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:48.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63340cd350 0x7f63340b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:48.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340b3e70 con 0x7f63340c3d80 2023-10-16T08:18:48.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340b1700 con 0x7f63340cd350 2023-10-16T08:18:48.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63340b19e0 con 0x7f63340ca3c0 2023-10-16T08:18:48.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 0x7f6334154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:48.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 0x7f6334154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55636/0 (socket says 172.21.15.32:55636) 2023-10-16T08:18:48.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 -- 172.21.15.32:0/853062295 learned_addr learned my addr 172.21.15.32:0/853062295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:48.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f6339945640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340ca3c0 0x7f63340b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:48.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a947640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63340cd350 0x7f63340b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:48.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 -- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340ca3c0 msgr2=0x7f63340b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340ca3c0 0x7f63340b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 -- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63340cd350 msgr2=0x7f63340b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:48.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63340cd350 0x7f63340b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:48.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 -- 172.21.15.32:0/853062295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009a70 con 0x7f63340c3d80 2023-10-16T08:18:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633a146640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 0x7f6334154490 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f633000c430 tx=0x7f633000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300052d0 con 0x7f63340c3d80 2023-10-16T08:18:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330005450 con 0x7f63340c3d80 2023-10-16T08:18:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330003840 con 0x7f63340c3d80 2023-10-16T08:18:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63340b1cc0 con 0x7f63340c3d80 2023-10-16T08:18:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.421+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340b2280 con 0x7f63340c3d80 2023-10-16T08:18:48.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.425+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633404e170 con 0x7f63340c3d80 2023-10-16T08:18:48.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.425+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63300039a0 con 0x7f63340c3d80 2023-10-16T08:18:48.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.425+0000 7f632affd640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63180664a0 0x7f6318068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:48.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.425+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5552+0+0 (secure 0 0 0) 0x7f6330083340 con 0x7f63340c3d80 2023-10-16T08:18:48.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.425+0000 7f6339945640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63180664a0 0x7f6318068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:48.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.425+0000 7f6339945640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63180664a0 0x7f6318068960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f63340c5650 tx=0x7f63240016c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.429+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6330050920 con 0x7f63340c3d80 2023-10-16T08:18:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:48.585+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6334071200 con 0x7f63340c3d80 2023-10-16T08:18:49.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.244+0000 7f632affd640 1 -- 172.21.15.32:0/853062295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'testcephfs1_data' created v52) v1 ==== 180+0+0 (secure 0 0 0) 0x7f6330053f40 con 0x7f63340c3d80 2023-10-16T08:18:49.253 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_data' created 2023-10-16T08:18:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63180664a0 msgr2=0x7f6318068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63180664a0 0x7f6318068960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f63340c5650 tx=0x7f63240016c0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 msgr2=0x7f6334154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 0x7f6334154490 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f633000c430 tx=0x7f633000c8f0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 shutdown_connections 2023-10-16T08:18:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63340cd350 0x7f63340b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f63180664a0 0x7f6318068960 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f63340ca3c0 0x7f63340b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 --2- 172.21.15.32:0/853062295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63340c3d80 0x7f6334154490 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 >> 172.21.15.32:0/853062295 conn(0x7f63340b0d10 msgr2=0x7f6334070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 shutdown_connections 2023-10-16T08:18:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.248+0000 7f633c3aa640 1 -- 172.21.15.32:0/853062295 wait complete. 2023-10-16T08:18:49.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testcephfs1 testcephfs1_metadata testcephfs1_data 2023-10-16T08:18:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.432+0000 7fbd0ad3b640 1 Processor -- start 2023-10-16T08:18:49.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.432+0000 7fbd0ad3b640 1 -- start start 2023-10-16T08:18:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd0ad3b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd040cf210 0x7fbd040cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd0ad3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd040ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd0ad3b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd040cfb30 con 0x7fbd0415cdc0 2023-10-16T08:18:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd0ad3b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd040cea10 con 0x7fbd040cf210 2023-10-16T08:18:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd0ad3b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040d0880 con 0x7fbd040d3320 2023-10-16T08:18:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd040cf210 0x7fbd040cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:49.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd040cf210 0x7fbd040cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55652/0 (socket says 172.21.15.32:55652) 2023-10-16T08:18:49.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 -- 172.21.15.32:0/4138791014 learned_addr learned my addr 172.21.15.32:0/4138791014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:49.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd0a53a640 1 --2- 172.21.15.32:0/4138791014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd040ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:49.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 -- 172.21.15.32:0/4138791014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 msgr2=0x7fbd040ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:49.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 --2- 172.21.15.32:0/4138791014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd040ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 -- 172.21.15.32:0/4138791014 >> v1:172.21.15.186:6789/0 conn(0x7fbd0415cdc0 legacy=0x7fbd040b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:49.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.436+0000 7fbd09d39640 1 -- 172.21.15.32:0/4138791014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd040b1c00 con 0x7fbd040cf210 2023-10-16T08:18:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd09d39640 1 --2- 172.21.15.32:0/4138791014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd040cf210 0x7fbd040cf5f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006fd0 tx=0x7fbd0001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420cec14b48c05c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:49.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd08d37640 1 -- 172.21.15.32:0/4138791014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000ab600 con 0x7fbd040cf210 2023-10-16T08:18:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd08d37640 1 -- 172.21.15.32:0/4138791014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd000ab760 con 0x7fbd040cf210 2023-10-16T08:18:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd08d37640 1 -- 172.21.15.32:0/4138791014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000abaf0 con 0x7fbd040cf210 2023-10-16T08:18:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/4138791014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd040cf210 msgr2=0x7fbd040cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/4138791014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd040cf210 0x7fbd040cf5f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006fd0 tx=0x7fbd0001c4f0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/4138791014 shutdown_connections 2023-10-16T08:18:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/4138791014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd040ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/4138791014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd040cf210 0x7fbd040cf5f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/4138791014 >> 172.21.15.32:0/4138791014 conn(0x7fbd040d2800 msgr2=0x7fbd040d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/4138791014 shutdown_connections 2023-10-16T08:18:49.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/4138791014 wait complete. 2023-10-16T08:18:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 Processor -- start 2023-10-16T08:18:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.440+0000 7fbd0ad3b640 1 -- start start 2023-10-16T08:18:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd040cf210 0x7fbd041560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd04154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 0x7fbd0406bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040c90f0 con 0x7fbd0415cdc0 2023-10-16T08:18:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040c8050 con 0x7fbd040cf210 2023-10-16T08:18:49.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040c8330 con 0x7fbd040d3320 2023-10-16T08:18:49.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 0x7fbd0406bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:49.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 0x7fbd0406bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55668/0 (socket says 172.21.15.32:55668) 2023-10-16T08:18:49.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 -- 172.21.15.32:0/686592910 learned_addr learned my addr 172.21.15.32:0/686592910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:49.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd09d39640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd040cf210 0x7fbd041560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd09538640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd04154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 -- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 msgr2=0x7fbd04154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd04154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 -- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd040cf210 msgr2=0x7fbd041560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd040cf210 0x7fbd041560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 -- 172.21.15.32:0/686592910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009480 con 0x7fbd0415cdc0 2023-10-16T08:18:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd09d39640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd040cf210 0x7fbd041560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd09538640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd04154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0a53a640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 0x7fbd0406bfe0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc007e40 tx=0x7fbcfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc014620 con 0x7fbd0415cdc0 2023-10-16T08:18:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00ee90 con 0x7fbd0415cdc0 2023-10-16T08:18:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc0149e0 con 0x7fbd0415cdc0 2023-10-16T08:18:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd040c8610 con 0x7fbd0415cdc0 2023-10-16T08:18:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd040c8bd0 con 0x7fbd0415cdc0 2023-10-16T08:18:49.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0404e9c0 con 0x7fbd0415cdc0 2023-10-16T08:18:49.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.444+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbcfc014b40 con 0x7fbd0415cdc0 2023-10-16T08:18:49.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.448+0000 7fbcfa7fc640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbce80664a0 0x7fbce8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:49.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.448+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fbcfc084980 con 0x7fbd0415cdc0 2023-10-16T08:18:49.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.448+0000 7fbd09d39640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbce80664a0 0x7fbce8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:49.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.448+0000 7fbd09d39640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbce80664a0 0x7fbce8068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd000af000 tx=0x7fbd00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:49.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.452+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbcfc051d10 con 0x7fbd0415cdc0 2023-10-16T08:18:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:49.640+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"} v 0) v1 -- 0x7fbd040b3f50 con 0x7fbd0415cdc0 2023-10-16T08:18:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 8 and data pool 9 2023-10-16T08:18:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.268+0000 7fbcfa7fc640 1 -- 172.21.15.32:0/686592910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"}]=0 new fs with metadata pool 8 and data pool 9 v24) v1 ==== 187+0+0 (secure 0 0 0) 0x7fbcfc055330 con 0x7fbd0415cdc0 2023-10-16T08:18:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.272+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbce80664a0 msgr2=0x7fbce8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.272+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbce80664a0 0x7fbce8068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd000af000 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2023-10-16T08:18:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.272+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 msgr2=0x7fbd0406bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.272+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 0x7fbd0406bfe0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc007e40 tx=0x7fbcfc00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 shutdown_connections 2023-10-16T08:18:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0415cdc0 0x7fbd0406bfe0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbce80664a0 0x7fbce8068960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd040d3320 0x7fbd04154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 --2- 172.21.15.32:0/686592910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd040cf210 0x7fbd041560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 >> 172.21.15.32:0/686592910 conn(0x7fbd040d2800 msgr2=0x7fbd0415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 shutdown_connections 2023-10-16T08:18:50.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.276+0000 7fbd0ad3b640 1 -- 172.21.15.32:0/686592910 wait complete. 2023-10-16T08:18:50.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.456+0000 7f54f1ed6640 1 Processor -- start 2023-10-16T08:18:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.456+0000 7f54f1ed6640 1 -- start start 2023-10-16T08:18:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54f1ed6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec15cb90 0x7f54ec0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54f1ed6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54ec0d15d0 0x7f54ec0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54f1ed6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec0ce770 con 0x7f54ec0d3010 2023-10-16T08:18:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54f1ed6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0d0600 con 0x7f54ec0d15d0 2023-10-16T08:18:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54f1ed6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0c6450 con 0x7f54ec15cb90 2023-10-16T08:18:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54ebfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54ec0d15d0 0x7f54ec0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54eaffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f54ec0d3010 0x7f54ec0d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37278/0 (socket says 172.21.15.32:37278) 2023-10-16T08:18:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54eaffd640 1 -- 172.21.15.32:0/553392828 learned_addr learned my addr 172.21.15.32:0/553392828 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54eb7fe640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec15cb90 0x7f54ec0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54ebfff640 1 -- 172.21.15.32:0/553392828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec15cb90 msgr2=0x7f54ec0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:50.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54ebfff640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec15cb90 0x7f54ec0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54ebfff640 1 -- 172.21.15.32:0/553392828 >> v1:172.21.15.32:6789/0 conn(0x7f54ec0d3010 legacy=0x7f54ec0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54ebfff640 1 -- 172.21.15.32:0/553392828 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0d08a0 con 0x7f54ec0d15d0 2023-10-16T08:18:50.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54ea7fc640 1 -- 172.21.15.32:0/553392828 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816827945 0 0) 0x7f54ec0ce770 con 0x7f54ec0d3010 2023-10-16T08:18:50.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.460+0000 7f54eb7fe640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec15cb90 0x7f54ec0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54ebfff640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec0d15d0 0x7f54ec0c6c10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009d90 tx=0x7f54e401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b45dc0801de41d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54ea7fc640 1 -- 172.21.15.32:0/553392828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40a5950 con 0x7f54ec0d15d0 2023-10-16T08:18:50.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54ea7fc640 1 -- 172.21.15.32:0/553392828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e40a5ab0 con 0x7f54ec0d15d0 2023-10-16T08:18:50.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54ea7fc640 1 -- 172.21.15.32:0/553392828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40a5e40 con 0x7f54ec0d15d0 2023-10-16T08:18:50.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 -- 172.21.15.32:0/553392828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec0d15d0 msgr2=0x7f54ec0c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec0d15d0 0x7f54ec0c6c10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009d90 tx=0x7f54e401c890 comp rx=0 tx=0).stop 2023-10-16T08:18:50.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 -- 172.21.15.32:0/553392828 shutdown_connections 2023-10-16T08:18:50.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec0d15d0 0x7f54ec0c6c10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/553392828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec15cb90 0x7f54ec0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 -- 172.21.15.32:0/553392828 >> 172.21.15.32:0/553392828 conn(0x7f54ec09bc50 msgr2=0x7f54ec0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:50.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 -- 172.21.15.32:0/553392828 shutdown_connections 2023-10-16T08:18:50.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 -- 172.21.15.32:0/553392828 wait complete. 2023-10-16T08:18:50.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 Processor -- start 2023-10-16T08:18:50.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.464+0000 7f54f1ed6640 1 -- start start 2023-10-16T08:18:50.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec0d15d0 0x7f54ec153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54ec0d3010 0x7f54ec150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 0x7f54ec0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec154660 con 0x7f54ec15cb90 2023-10-16T08:18:50.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0b62a0 con 0x7f54ec0d3010 2023-10-16T08:18:50.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1548b0 con 0x7f54ec0d15d0 2023-10-16T08:18:50.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 0x7f54ec0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 0x7f54ec0b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55688/0 (socket says 172.21.15.32:55688) 2023-10-16T08:18:50.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 -- 172.21.15.32:0/2889987466 learned_addr learned my addr 172.21.15.32:0/2889987466 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:50.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54eb7fe640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec0d15d0 0x7f54ec153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54eaffd640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54ec0d3010 0x7f54ec150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 -- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec0d15d0 msgr2=0x7f54ec153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec0d15d0 0x7f54ec153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 -- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54ec0d3010 msgr2=0x7f54ec150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54ec0d3010 0x7f54ec150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 -- 172.21.15.32:0/2889987466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e4009a70 con 0x7f54ec15cb90 2023-10-16T08:18:50.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54ebfff640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 0x7f54ec0b5d60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f54e40b0040 tx=0x7f54e4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:50.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40ade40 con 0x7f54ec15cb90 2023-10-16T08:18:50.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e4009450 con 0x7f54ec15cb90 2023-10-16T08:18:50.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40b9670 con 0x7f54ec15cb90 2023-10-16T08:18:50.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec154b90 con 0x7f54ec15cb90 2023-10-16T08:18:50.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec155150 con 0x7f54ec15cb90 2023-10-16T08:18:50.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.468+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec155340 con 0x7f54ec15cb90 2023-10-16T08:18:50.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.472+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f54e40b97d0 con 0x7f54ec15cb90 2023-10-16T08:18:50.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.472+0000 7f54d3fff640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54d80664a0 0x7f54d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.472+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f54e411f7f0 con 0x7f54ec15cb90 2023-10-16T08:18:50.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.472+0000 7f54eb7fe640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54d80664a0 0x7f54d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.472+0000 7f54eb7fe640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54d80664a0 0x7f54d8068960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f54ec06c260 tx=0x7f54dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.476+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f54e40ecc40 con 0x7f54ec15cb90 2023-10-16T08:18:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.632+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54ec074010 con 0x7f54ec15cb90 2023-10-16T08:18:50.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.632+0000 7f54d3fff640 1 -- 172.21.15.32:0/2889987466 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16803 (secure 0 0 0) 0x7f54e40f0260 con 0x7f54ec15cb90 2023-10-16T08:18:50.640 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:50.640 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:49.647765+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54d80664a0 msgr2=0x7f54d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54d80664a0 0x7f54d8068960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f54ec06c260 tx=0x7f54dc008040 comp rx=0 tx=0).stop 2023-10-16T08:18:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 msgr2=0x7f54ec0b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 0x7f54ec0b5d60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f54e40b0040 tx=0x7f54e4019630 comp rx=0 tx=0).stop 2023-10-16T08:18:50.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 shutdown_connections 2023-10-16T08:18:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ec15cb90 0x7f54ec0b5d60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54ec0d3010 0x7f54ec150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f54d80664a0 0x7f54d8068960 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 --2- 172.21.15.32:0/2889987466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f54ec0d15d0 0x7f54ec153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 >> 172.21.15.32:0/2889987466 conn(0x7f54ec09bc50 msgr2=0x7f54ec0ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 shutdown_connections 2023-10-16T08:18:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.636+0000 7f54f1ed6640 1 -- 172.21.15.32:0/2889987466 wait complete. 2023-10-16T08:18:50.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:50.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.796+0000 7f728833e640 1 Processor -- start 2023-10-16T08:18:50.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.796+0000 7f728833e640 1 -- start start 2023-10-16T08:18:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f728833e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7280070c40 0x7f728015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f728833e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f72800b07f0 0x7f7280164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f728833e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728015be60 con 0x7f72800b4020 2023-10-16T08:18:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f728833e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728015b3b0 con 0x7f72800b07f0 2023-10-16T08:18:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f728833e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728015c1f0 con 0x7f7280070c40 2023-10-16T08:18:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72858d9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72800b4020 0x7f72800b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37294/0 (socket says 172.21.15.32:37294) 2023-10-16T08:18:50.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72858d9640 1 -- 172.21.15.32:0/3105223257 learned_addr learned my addr 172.21.15.32:0/3105223257 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72868db640 1 --2- 172.21.15.32:0/3105223257 >> v2:172.21.15.186:3300/0 conn(0x7f72800b07f0 0x7f7280164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72860da640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7280070c40 0x7f728015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72850d8640 1 -- 172.21.15.32:0/3105223257 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039856885 0 0) 0x7f728015be60 con 0x7f72800b4020 2023-10-16T08:18:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72850d8640 1 -- 172.21.15.32:0/3105223257 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f726c003530 con 0x7f72800b4020 2023-10-16T08:18:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72868db640 1 -- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7280070c40 msgr2=0x7f728015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72868db640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7280070c40 0x7f728015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72868db640 1 -- 172.21.15.32:0/3105223257 >> v1:172.21.15.32:6789/0 conn(0x7f72800b4020 legacy=0x7f72800b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72868db640 1 -- 172.21.15.32:0/3105223257 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280071020 con 0x7f72800b07f0 2023-10-16T08:18:50.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.800+0000 7f72860da640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7280070c40 0x7f728015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f72868db640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72800b07f0 0x7f7280164490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7278009d90 tx=0x7f727801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4078afc66512ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f72850d8640 1 -- 172.21.15.32:0/3105223257 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780a5980 con 0x7f72800b07f0 2023-10-16T08:18:50.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f72850d8640 1 -- 172.21.15.32:0/3105223257 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72780a5ae0 con 0x7f72800b07f0 2023-10-16T08:18:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f72850d8640 1 -- 172.21.15.32:0/3105223257 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780a5e70 con 0x7f72800b07f0 2023-10-16T08:18:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 -- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72800b07f0 msgr2=0x7f7280164490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72800b07f0 0x7f7280164490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7278009d90 tx=0x7f727801c8c0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 -- 172.21.15.32:0/3105223257 shutdown_connections 2023-10-16T08:18:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72800b07f0 0x7f7280164490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 --2- 172.21.15.32:0/3105223257 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7280070c40 0x7f728015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 -- 172.21.15.32:0/3105223257 >> 172.21.15.32:0/3105223257 conn(0x7f728006c190 msgr2=0x7f728006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 -- 172.21.15.32:0/3105223257 shutdown_connections 2023-10-16T08:18:50.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 -- 172.21.15.32:0/3105223257 wait complete. 2023-10-16T08:18:50.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.804+0000 7f728833e640 1 Processor -- start 2023-10-16T08:18:50.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- start start 2023-10-16T08:18:50.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7280070c40 0x7f7280150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72800b07f0 0x7f728014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 0x7f728014b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728014be90 con 0x7f72800b4020 2023-10-16T08:18:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728014ec10 con 0x7f7280070c40 2023-10-16T08:18:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728014f970 con 0x7f72800b07f0 2023-10-16T08:18:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 0x7f728014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 0x7f728014b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55704/0 (socket says 172.21.15.32:55704) 2023-10-16T08:18:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 -- 172.21.15.32:0/3324633020 learned_addr learned my addr 172.21.15.32:0/3324633020 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72858d9640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72800b07f0 0x7f728014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72860da640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7280070c40 0x7f7280150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 -- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72800b07f0 msgr2=0x7f728014b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72800b07f0 0x7f728014b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 -- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7280070c40 msgr2=0x7f7280150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7280070c40 0x7f7280150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 -- 172.21.15.32:0/3324633020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c009420 con 0x7f72800b4020 2023-10-16T08:18:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f72868db640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 0x7f728014b950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f72780b0040 tx=0x7f7278019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780ad850 con 0x7f72800b4020 2023-10-16T08:18:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72780ad9b0 con 0x7f72800b4020 2023-10-16T08:18:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7278009a70 con 0x7f72800b4020 2023-10-16T08:18:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780add60 con 0x7f72800b4020 2023-10-16T08:18:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728014f470 con 0x7f72800b4020 2023-10-16T08:18:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.808+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280074010 con 0x7f72800b4020 2023-10-16T08:18:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.812+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f72780b4050 con 0x7f72800b4020 2023-10-16T08:18:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.812+0000 7f7276ffd640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72640664a0 0x7f7264068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.812+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f727811eda0 con 0x7f72800b4020 2023-10-16T08:18:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.812+0000 7f72860da640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72640664a0 0x7f7264068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.812+0000 7f72860da640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72640664a0 0x7f7264068960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f727c009cf0 tx=0x7f727c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.816+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f72780ec160 con 0x7f72800b4020 2023-10-16T08:18:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.972+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72800b3880 con 0x7f72800b4020 2023-10-16T08:18:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.972+0000 7f7276ffd640 1 -- 172.21.15.32:0/3324633020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16803 (secure 0 0 0) 0x7f72780b2090 con 0x7f72800b4020 2023-10-16T08:18:50.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:50.981 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:49.647765+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72640664a0 msgr2=0x7f7264068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72640664a0 0x7f7264068960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f727c009cf0 tx=0x7f727c009190 comp rx=0 tx=0).stop 2023-10-16T08:18:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 msgr2=0x7f728014b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 0x7f728014b950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f72780b0040 tx=0x7f7278019660 comp rx=0 tx=0).stop 2023-10-16T08:18:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 shutdown_connections 2023-10-16T08:18:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72800b4020 0x7f728014b950 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f72640664a0 0x7f7264068960 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f72800b07f0 0x7f728014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 --2- 172.21.15.32:0/3324633020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7280070c40 0x7f7280150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 >> 172.21.15.32:0/3324633020 conn(0x7f728006c190 msgr2=0x7f72800b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 shutdown_connections 2023-10-16T08:18:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:50.976+0000 7f728833e640 1 -- 172.21.15.32:0/3324633020 wait complete. 2023-10-16T08:18:51.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 standby_count_wanted 0 2023-10-16T08:18:51.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.140+0000 7fb6ff29c640 1 Processor -- start 2023-10-16T08:18:51.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.140+0000 7fb6ff29c640 1 -- start start 2023-10-16T08:18:51.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6ff29c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6f815cba0 0x7fb6f80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:51.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6ff29c640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fb6f80b84e0 0x7fb6f80cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:51.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6ff29c640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80d1670 con 0x7fb6f80d0040 2023-10-16T08:18:51.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6ff29c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f80d1460 con 0x7fb6f815cba0 2023-10-16T08:18:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6ff29c640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f80d04b0 con 0x7fb6f80b84e0 2023-10-16T08:18:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6f815cba0 0x7fb6f80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6f815cba0 0x7fb6f80d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55716/0 (socket says 172.21.15.32:55716) 2023-10-16T08:18:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 -- 172.21.15.32:0/1987729602 learned_addr learned my addr 172.21.15.32:0/1987729602 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fc837640 1 --2- 172.21.15.32:0/1987729602 >> v2:172.21.15.186:3300/0 conn(0x7fb6f80b84e0 0x7fb6f80cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 -- 172.21.15.32:0/1987729602 >> v2:172.21.15.186:3300/0 conn(0x7fb6f80b84e0 msgr2=0x7fb6f80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 --2- 172.21.15.32:0/1987729602 >> v2:172.21.15.186:3300/0 conn(0x7fb6f80b84e0 0x7fb6f80cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:51.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 -- 172.21.15.32:0/1987729602 >> v1:172.21.15.186:6789/0 conn(0x7fb6f80d0040 legacy=0x7fb6f80d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:51.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fd038640 1 -- 172.21.15.32:0/1987729602 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f80d1950 con 0x7fb6f815cba0 2023-10-16T08:18:51.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.144+0000 7fb6fc837640 1 --2- 172.21.15.32:0/1987729602 >> v2:172.21.15.186:3300/0 conn(0x7fb6f80b84e0 0x7fb6f80cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:51.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6fd038640 1 --2- 172.21.15.32:0/1987729602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f815cba0 0x7fb6f80d0f20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80cf2f0 tx=0x7fb6f401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d2efdf1dbb1d69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6effff640 1 -- 172.21.15.32:0/1987729602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4004230 con 0x7fb6f815cba0 2023-10-16T08:18:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6effff640 1 -- 172.21.15.32:0/1987729602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f40a7050 con 0x7fb6f815cba0 2023-10-16T08:18:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6effff640 1 -- 172.21.15.32:0/1987729602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4003620 con 0x7fb6f815cba0 2023-10-16T08:18:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 -- 172.21.15.32:0/1987729602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f815cba0 msgr2=0x7fb6f80d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/1987729602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f815cba0 0x7fb6f80d0f20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80cf2f0 tx=0x7fb6f401c430 comp rx=0 tx=0).stop 2023-10-16T08:18:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 -- 172.21.15.32:0/1987729602 shutdown_connections 2023-10-16T08:18:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/1987729602 >> v2:172.21.15.186:3300/0 conn(0x7fb6f80b84e0 0x7fb6f80cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/1987729602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f815cba0 0x7fb6f80d0f20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 -- 172.21.15.32:0/1987729602 >> 172.21.15.32:0/1987729602 conn(0x7fb6f80719c0 msgr2=0x7fb6f8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 -- 172.21.15.32:0/1987729602 shutdown_connections 2023-10-16T08:18:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 -- 172.21.15.32:0/1987729602 wait complete. 2023-10-16T08:18:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 Processor -- start 2023-10-16T08:18:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.148+0000 7fb6ff29c640 1 -- start start 2023-10-16T08:18:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6f80b84e0 0x7fb6f80c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 0x7fb6f814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:51.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6f815cba0 0x7fb6f80c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f814fec0 con 0x7fb6f80d0040 2023-10-16T08:18:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8157910 con 0x7fb6f80b84e0 2023-10-16T08:18:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8159010 con 0x7fb6f815cba0 2023-10-16T08:18:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 0x7fb6f814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:51.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 0x7fb6f814f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55732/0 (socket says 172.21.15.32:55732) 2023-10-16T08:18:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 -- 172.21.15.32:0/3864959349 learned_addr learned my addr 172.21.15.32:0/3864959349 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fd038640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6f80b84e0 0x7fb6f80c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fd839640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6f815cba0 0x7fb6f80c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 -- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6f815cba0 msgr2=0x7fb6f80c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6f815cba0 0x7fb6f80c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 -- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6f80b84e0 msgr2=0x7fb6f80c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6f80b84e0 0x7fb6f80c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 -- 172.21.15.32:0/3864959349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40094d0 con 0x7fb6f80d0040 2023-10-16T08:18:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6fc837640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 0x7fb6f814f540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c7f0 tx=0x7fb6e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800cea0 con 0x7fb6f80d0040 2023-10-16T08:18:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e8007a30 con 0x7fb6f80d0040 2023-10-16T08:18:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f81592f0 con 0x7fb6f80d0040 2023-10-16T08:18:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8007de0 con 0x7fb6f80d0040 2023-10-16T08:18:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f80ca160 con 0x7fb6f80d0040 2023-10-16T08:18:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f80c9520 con 0x7fb6f80d0040 2023-10-16T08:18:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.152+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb6e80113f0 con 0x7fb6f80d0040 2023-10-16T08:18:51.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.156+0000 7fb6edffb640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb6dc0664a0 0x7fb6dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.156+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fb6e8083f70 con 0x7fb6f80d0040 2023-10-16T08:18:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.156+0000 7fb6fd038640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb6dc0664a0 0x7fb6dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.156+0000 7fb6fd038640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb6dc0664a0 0x7fb6dc068960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40ad000 tx=0x7fb6f40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:51.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.160+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb6e80b5090 con 0x7fb6f80d0040 2023-10-16T08:18:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:51.352+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb6f804e170 con 0x7fb6f80d0040 2023-10-16T08:18:52.240 INFO:tasks.ceph.mds.c.smithi032.stderr:2023-10-16T08:18:52.236+0000 7fb5056e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:18:52.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6edffb640 1 -- 172.21.15.32:0/3864959349 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb6e80513d0 con 0x7fb6f80d0040 2023-10-16T08:18:52.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb6dc0664a0 msgr2=0x7fb6dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb6dc0664a0 0x7fb6dc068960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40ad000 tx=0x7fb6f40a6000 comp rx=0 tx=0).stop 2023-10-16T08:18:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 msgr2=0x7fb6f814f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 0x7fb6f814f540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c7f0 tx=0x7fb6e800ccb0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 shutdown_connections 2023-10-16T08:18:52.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb6dc0664a0 0x7fb6dc068960 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb6f815cba0 0x7fb6f80c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80d0040 0x7fb6f814f540 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 --2- 172.21.15.32:0/3864959349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6f80b84e0 0x7fb6f80c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 >> 172.21.15.32:0/3864959349 conn(0x7fb6f80719c0 msgr2=0x7fb6f8159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:52.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 shutdown_connections 2023-10-16T08:18:52.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.280+0000 7fb6ff29c640 1 -- 172.21.15.32:0/3864959349 wait complete. 2023-10-16T08:18:52.311 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T08:18:52.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 allow_standby_replay false 2023-10-16T08:18:52.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.456+0000 7f7eaf9d0640 1 Processor -- start 2023-10-16T08:18:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.456+0000 7f7eaf9d0640 1 -- start start 2023-10-16T08:18:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.456+0000 7f7eaf9d0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 0x7f7ea80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.456+0000 7f7eaf9d0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea80d1670 con 0x7f7ea815cba0 2023-10-16T08:18:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eaf9d0640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea80d1950 con 0x7f7ea80d0210 2023-10-16T08:18:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eaf9d0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80d1460 con 0x7f7ea80b84e0 2023-10-16T08:18:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7ead76c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7ea815cba0 0x7f7ea80cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37306/0 (socket says 172.21.15.32:37306) 2023-10-16T08:18:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7ead76c640 1 -- 172.21.15.32:0/1154476949 learned_addr learned my addr 172.21.15.32:0/1154476949 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eadf6d640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 0x7f7ea80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607240759 0 0) 0x7f7ea80d1670 con 0x7f7ea815cba0 2023-10-16T08:18:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e94003530 con 0x7f7ea815cba0 2023-10-16T08:18:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428757178 0 0) 0x7f7e94003530 con 0x7f7ea815cba0 2023-10-16T08:18:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 msgr2=0x7f7ea80d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 0x7f7ea80d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 >> v1:172.21.15.186:6789/0 conn(0x7f7ea80d0210 legacy=0x7f7ea80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80d1c30 con 0x7f7ea815cba0 2023-10-16T08:18:52.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f7ea4001680 con 0x7f7ea815cba0 2023-10-16T08:18:52.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 >> v1:172.21.15.32:6789/0 conn(0x7f7ea815cba0 legacy=0x7f7ea80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 0x7f7e940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e94003ef0 0x7f7e940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94004d10 0x7f7e94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80d1460 con 0x7f7e94003ef0 2023-10-16T08:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9400ba40 con 0x7f7e94004d10 2023-10-16T08:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9400bc00 con 0x7f7ea80b84e0 2023-10-16T08:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e94003ef0 0x7f7e940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7ead76c640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 0x7f7e940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eadf6d640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94004d10 0x7f7e94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 msgr2=0x7f7e940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80b84e0 0x7f7e940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94004d10 msgr2=0x7f7e94005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94004d10 0x7f7e94005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.460+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1154476949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e94003530 con 0x7f7e94003ef0 2023-10-16T08:18:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e94003ef0 0x7f7e940047d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80b5aa0 tx=0x7f7e9801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206d988cf4bea2ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980a8830 con 0x7f7e94003ef0 2023-10-16T08:18:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e980a8990 con 0x7f7e94003ef0 2023-10-16T08:18:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7e9ffff640 1 -- 172.21.15.32:0/1154476949 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980a8d40 con 0x7f7e94003ef0 2023-10-16T08:18:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1154476949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e94003ef0 msgr2=0x7f7e940047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1154476949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e94003ef0 0x7f7e940047d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80b5aa0 tx=0x7f7e9801ec10 comp rx=0 tx=0).stop 2023-10-16T08:18:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eadf6d640 1 -- 172.21.15.32:0/1154476949 reap_dead start 2023-10-16T08:18:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1154476949 shutdown_connections 2023-10-16T08:18:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1154476949 >> 172.21.15.32:0/1154476949 conn(0x7f7ea80719c0 msgr2=0x7f7ea8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1154476949 shutdown_connections 2023-10-16T08:18:52.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1154476949 wait complete. 2023-10-16T08:18:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 Processor -- start 2023-10-16T08:18:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.464+0000 7f7eaf9d0640 1 -- start start 2023-10-16T08:18:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94003ef0 0x7f7ea8149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 0x7f7ea80d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80d3020 0x7f7ea815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea814a520 con 0x7f7ea815ec00 2023-10-16T08:18:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80d34a0 con 0x7f7e94003ef0 2023-10-16T08:18:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81f64f0 con 0x7f7ea80d3020 2023-10-16T08:18:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 0x7f7ea80d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 0x7f7ea80d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55756/0 (socket says 172.21.15.32:55756) 2023-10-16T08:18:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1035628310 learned_addr learned my addr 172.21.15.32:0/1035628310 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eadf6d640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80d3020 0x7f7ea815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7ead76c640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94003ef0 0x7f7ea8149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80d3020 msgr2=0x7f7ea815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80d3020 0x7f7ea815cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94003ef0 msgr2=0x7f7ea8149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:52.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94003ef0 0x7f7ea8149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:52.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 -- 172.21.15.32:0/1035628310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea81f5970 con 0x7f7ea815ec00 2023-10-16T08:18:52.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7ead76c640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94003ef0 0x7f7ea8149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:52.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eacf6b640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 0x7f7ea80d2ae0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c10 tx=0x7f7e94005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98005310 con 0x7f7ea815ec00 2023-10-16T08:18:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98005470 con 0x7f7ea815ec00 2023-10-16T08:18:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980069e0 con 0x7f7ea815ec00 2023-10-16T08:18:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea81f5c50 con 0x7f7ea815ec00 2023-10-16T08:18:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea81f6210 con 0x7f7ea815ec00 2023-10-16T08:18:52.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.468+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea804e170 con 0x7f7ea815ec00 2023-10-16T08:18:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.472+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7e980b7040 con 0x7f7ea815ec00 2023-10-16T08:18:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.472+0000 7f7e9dffb640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e8c0664a0 0x7f7e8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.472+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f7e98120500 con 0x7f7ea815ec00 2023-10-16T08:18:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.472+0000 7f7ead76c640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e8c0664a0 0x7f7e8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:52.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.472+0000 7f7ead76c640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e8c0664a0 0x7f7e8c068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4007bd0 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:52.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.476+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7e980eda80 con 0x7f7ea815ec00 2023-10-16T08:18:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:52.664+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7ea815b250 con 0x7f7ea815ec00 2023-10-16T08:18:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.284+0000 7f7e9dffb640 1 -- 172.21.15.32:0/1035628310 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7e980a87f0 con 0x7f7ea815ec00 2023-10-16T08:18:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e8c0664a0 msgr2=0x7f7e8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e8c0664a0 0x7f7e8c068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4007bd0 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2023-10-16T08:18:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 msgr2=0x7f7ea80d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 0x7f7ea80d2ae0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c10 tx=0x7f7e94005800 comp rx=0 tx=0).stop 2023-10-16T08:18:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 shutdown_connections 2023-10-16T08:18:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e8c0664a0 0x7f7e8c068960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7ea80d3020 0x7f7ea815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea815ec00 0x7f7ea80d2ae0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 --2- 172.21.15.32:0/1035628310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e94003ef0 0x7f7ea8149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 >> 172.21.15.32:0/1035628310 conn(0x7f7ea80719c0 msgr2=0x7f7ea80b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 shutdown_connections 2023-10-16T08:18:53.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.288+0000 7f7eaf9d0640 1 -- 172.21.15.32:0/1035628310 wait complete. 2023-10-16T08:18:53.318 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'testcephfs1' 2023-10-16T08:18:53.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:53.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 Processor -- start 2023-10-16T08:18:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 -- start start 2023-10-16T08:18:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e940d3010 0x7f2e940d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2e940d1400 0x7f2e940cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e940d1890 con 0x7f2e940d3010 2023-10-16T08:18:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e940ce770 con 0x7f2e940d1400 2023-10-16T08:18:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e9b26b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940d0600 con 0x7f2e9415cb90 2023-10-16T08:18:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e940d3010 0x7f2e940d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e940d3010 0x7f2e940d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43528/0 (socket says 172.21.15.32:43528) 2023-10-16T08:18:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 -- 172.21.15.32:0/1712868326 learned_addr learned my addr 172.21.15.32:0/1712868326 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:53.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e99808640 1 --2- 172.21.15.32:0/1712868326 >> v2:172.21.15.186:3300/0 conn(0x7f2e940d1400 0x7f2e940cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e99007640 1 --2- 172.21.15.32:0/1712868326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 -- 172.21.15.32:0/1712868326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 msgr2=0x7f2e940cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 --2- 172.21.15.32:0/1712868326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 -- 172.21.15.32:0/1712868326 >> v2:172.21.15.186:3300/0 conn(0x7f2e940d1400 msgr2=0x7f2e940cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 --2- 172.21.15.32:0/1712868326 >> v2:172.21.15.186:3300/0 conn(0x7f2e940d1400 0x7f2e940cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.460+0000 7f2e98806640 1 -- 172.21.15.32:0/1712868326 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940c6450 con 0x7f2e940d3010 2023-10-16T08:18:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e98806640 1 --2- 172.21.15.32:0/1712868326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e940d0ec0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2e940725c0 tx=0x7f2e8401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfacafe16610393 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e8bfff640 1 -- 172.21.15.32:0/1712868326 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e840ab600 con 0x7f2e940d3010 2023-10-16T08:18:53.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e8bfff640 1 -- 172.21.15.32:0/1712868326 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e840ab760 con 0x7f2e940d3010 2023-10-16T08:18:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e8bfff640 1 -- 172.21.15.32:0/1712868326 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e840abaf0 con 0x7f2e940d3010 2023-10-16T08:18:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 -- 172.21.15.32:0/1712868326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 msgr2=0x7f2e940d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/1712868326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e940d0ec0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2e940725c0 tx=0x7f2e8401c440 comp rx=0 tx=0).stop 2023-10-16T08:18:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 -- 172.21.15.32:0/1712868326 shutdown_connections 2023-10-16T08:18:53.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/1712868326 >> v2:172.21.15.186:3300/0 conn(0x7f2e940d1400 0x7f2e940cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/1712868326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e940d0ec0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/1712868326 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 -- 172.21.15.32:0/1712868326 >> 172.21.15.32:0/1712868326 conn(0x7f2e9409bc50 msgr2=0x7f2e940d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 -- 172.21.15.32:0/1712868326 shutdown_connections 2023-10-16T08:18:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.464+0000 7f2e9b26b640 1 -- 172.21.15.32:0/1712868326 wait complete. 2023-10-16T08:18:53.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 Processor -- start 2023-10-16T08:18:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 -- start start 2023-10-16T08:18:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e940d1400 0x7f2e94149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e9415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9415c3b0 con 0x7f2e940d3010 2023-10-16T08:18:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e94149bc0 con 0x7f2e940d1400 2023-10-16T08:18:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e94154c00 con 0x7f2e9415cb90 2023-10-16T08:18:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e9415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e9415b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43534/0 (socket says 172.21.15.32:43534) 2023-10-16T08:18:53.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 -- 172.21.15.32:0/60757450 learned_addr learned my addr 172.21.15.32:0/60757450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e99808640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e99007640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e940d1400 0x7f2e94149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 -- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 msgr2=0x7f2e940b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 -- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e940d1400 msgr2=0x7f2e94149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e940d1400 0x7f2e94149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 -- 172.21.15.32:0/60757450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840094d0 con 0x7f2e940d3010 2023-10-16T08:18:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e98806640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e9415b5f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2e8401aaf0 tx=0x7f2e8401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e840be5f0 con 0x7f2e940d3010 2023-10-16T08:18:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e840be750 con 0x7f2e940d3010 2023-10-16T08:18:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e840beb00 con 0x7f2e940d3010 2023-10-16T08:18:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.468+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9415c0d0 con 0x7f2e940d3010 2023-10-16T08:18:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e94154400 con 0x7f2e940d3010 2023-10-16T08:18:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e94153f80 con 0x7f2e940d3010 2023-10-16T08:18:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2e840b43f0 con 0x7f2e940d3010 2023-10-16T08:18:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e89ffb640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2e780664a0 0x7f2e78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f2e8411eca0 con 0x7f2e940d3010 2023-10-16T08:18:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e99007640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2e780664a0 0x7f2e78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.472+0000 7f2e99007640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2e780664a0 0x7f2e78068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009a20 tx=0x7f2e90006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.476+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2e94153f80 con 0x7f2e940d3010 2023-10-16T08:18:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.664+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e94153f80 con 0x7f2e940d3010 2023-10-16T08:18:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.664+0000 7f2e89ffb640 1 -- 172.21.15.32:0/60757450 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f2e94153f80 con 0x7f2e940d3010 2023-10-16T08:18:53.672 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:53.672 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:18:49.648104+0000","modified":"2023-10-16T08:18:52.672446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/3873719355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3873719355},{"type":"v1","addr":"172.21.15.32:6834","nonce":3873719355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T08:18:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2023-10-16T08:18:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2e780664a0 msgr2=0x7f2e78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2e780664a0 0x7f2e78068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009a20 tx=0x7f2e90006f90 comp rx=0 tx=0).stop 2023-10-16T08:18:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 msgr2=0x7f2e9415b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e9415b5f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2e8401aaf0 tx=0x7f2e8401ac80 comp rx=0 tx=0).stop 2023-10-16T08:18:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 shutdown_connections 2023-10-16T08:18:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2e780664a0 0x7f2e78068960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2e9415cb90 0x7f2e940b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e940d3010 0x7f2e9415b5f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 --2- 172.21.15.32:0/60757450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e940d1400 0x7f2e94149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 >> 172.21.15.32:0/60757450 conn(0x7f2e9409bc50 msgr2=0x7f2e940c7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 shutdown_connections 2023-10-16T08:18:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.668+0000 7f2e9b26b640 1 -- 172.21.15.32:0/60757450 wait complete. 2023-10-16T08:18:53.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:53.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 Processor -- start 2023-10-16T08:18:53.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 -- start start 2023-10-16T08:18:53.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4e6c15c910 0x7f4e6c0d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e6c0d3170 0x7f4e6c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0ce730 con 0x7f4e6c0cefe0 2023-10-16T08:18:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0d17a0 con 0x7f4e6c0d3170 2023-10-16T08:18:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e70cad640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0d0dd0 con 0x7f4e6c15c910 2023-10-16T08:18:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e6c0cefe0 0x7f4e6c06b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42086/0 (socket says 172.21.15.32:42086) 2023-10-16T08:18:53.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6bfff640 1 -- 172.21.15.32:0/3248576703 learned_addr learned my addr 172.21.15.32:0/3248576703 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:53.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6affd640 1 --2- 172.21.15.32:0/3248576703 >> v2:172.21.15.32:3300/0 conn(0x7f4e6c0d3170 0x7f4e6c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6b7fe640 1 --2- 172.21.15.32:0/3248576703 >> v2:172.21.15.186:3300/0 conn(0x7f4e6c15c910 0x7f4e6c0d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6affd640 1 -- 172.21.15.32:0/3248576703 >> v2:172.21.15.186:3300/0 conn(0x7f4e6c15c910 msgr2=0x7f4e6c0d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:53.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6affd640 1 --2- 172.21.15.32:0/3248576703 >> v2:172.21.15.186:3300/0 conn(0x7f4e6c15c910 0x7f4e6c0d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6affd640 1 -- 172.21.15.32:0/3248576703 >> v1:172.21.15.32:6789/0 conn(0x7f4e6c0cefe0 legacy=0x7f4e6c06b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6a7fc640 1 -- 172.21.15.32:0/3248576703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468038543 0 0) 0x7f4e6c0ce730 con 0x7f4e6c0cefe0 2023-10-16T08:18:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.832+0000 7f4e6affd640 1 -- 172.21.15.32:0/3248576703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0d04a0 con 0x7f4e6c0d3170 2023-10-16T08:18:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e6affd640 1 --2- 172.21.15.32:0/3248576703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c0cde30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c08b8d0 tx=0x7f4e6001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb938b0238cc4a6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e6a7fc640 1 -- 172.21.15.32:0/3248576703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e600ac600 con 0x7f4e6c0d3170 2023-10-16T08:18:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e6a7fc640 1 -- 172.21.15.32:0/3248576703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e600a8050 con 0x7f4e6c0d3170 2023-10-16T08:18:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e6a7fc640 1 -- 172.21.15.32:0/3248576703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e600ac980 con 0x7f4e6c0d3170 2023-10-16T08:18:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 -- 172.21.15.32:0/3248576703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 msgr2=0x7f4e6c0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 --2- 172.21.15.32:0/3248576703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c0cde30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c08b8d0 tx=0x7f4e6001c3a0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 -- 172.21.15.32:0/3248576703 shutdown_connections 2023-10-16T08:18:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 --2- 172.21.15.32:0/3248576703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c0cde30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 --2- 172.21.15.32:0/3248576703 >> v2:172.21.15.186:3300/0 conn(0x7f4e6c15c910 0x7f4e6c0d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 -- 172.21.15.32:0/3248576703 >> 172.21.15.32:0/3248576703 conn(0x7f4e6c09bb50 msgr2=0x7f4e6c0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 -- 172.21.15.32:0/3248576703 shutdown_connections 2023-10-16T08:18:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 -- 172.21.15.32:0/3248576703 wait complete. 2023-10-16T08:18:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 Processor -- start 2023-10-16T08:18:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.836+0000 7f4e70cad640 1 -- start start 2023-10-16T08:18:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c0cefe0 0x7f4e6c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c15c910 0x7f4e6c154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c156590 con 0x7f4e6c0d3170 2023-10-16T08:18:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c156870 con 0x7f4e6c0cefe0 2023-10-16T08:18:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c156b50 con 0x7f4e6c15c910 2023-10-16T08:18:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c15aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43546/0 (socket says 172.21.15.32:43546) 2023-10-16T08:18:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 -- 172.21.15.32:0/1542759399 learned_addr learned my addr 172.21.15.32:0/1542759399 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6b7fe640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c0cefe0 0x7f4e6c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6bfff640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c15c910 0x7f4e6c154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 -- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c15c910 msgr2=0x7f4e6c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c15c910 0x7f4e6c154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 -- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c0cefe0 msgr2=0x7f4e6c157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c0cefe0 0x7f4e6c157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 -- 172.21.15.32:0/1542759399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e60009420 con 0x7f4e6c0d3170 2023-10-16T08:18:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6b7fe640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c0cefe0 0x7f4e6c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e6affd640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c15aa90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4e60019fd0 tx=0x7f4e6001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e600b7620 con 0x7f4e6c0d3170 2023-10-16T08:18:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e600b77a0 con 0x7f4e6c0d3170 2023-10-16T08:18:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c156e30 con 0x7f4e6c0d3170 2023-10-16T08:18:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e600b7b50 con 0x7f4e6c0d3170 2023-10-16T08:18:53.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c15afd0 con 0x7f4e6c0d3170 2023-10-16T08:18:53.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.840+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c04e9c0 con 0x7f4e6c0d3170 2023-10-16T08:18:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.848+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4e600b3040 con 0x7f4e6c0d3170 2023-10-16T08:18:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.848+0000 7f4e53fff640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4e580664a0 0x7f4e58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.848+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f4e6011ded0 con 0x7f4e6c0d3170 2023-10-16T08:18:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.848+0000 7f4e6b7fe640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4e580664a0 0x7f4e58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.848+0000 7f4e6b7fe640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4e580664a0 0x7f4e58068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006fd0 tx=0x7f4e5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:53.848+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4e600eb0c0 con 0x7f4e6c0d3170 2023-10-16T08:18:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.004+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e6c1572e0 con 0x7f4e6c0d3170 2023-10-16T08:18:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e53fff640 1 -- 172.21.15.32:0/1542759399 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f4e600b1090 con 0x7f4e6c0d3170 2023-10-16T08:18:54.014 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:54.014 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:54.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4e580664a0 msgr2=0x7f4e58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4e580664a0 0x7f4e58068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006fd0 tx=0x7f4e5c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 msgr2=0x7f4e6c15aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c15aa90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4e60019fd0 tx=0x7f4e6001abe0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 shutdown_connections 2023-10-16T08:18:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4e580664a0 0x7f4e58068960 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4e6c15c910 0x7f4e6c154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e6c0d3170 0x7f4e6c15aa90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 --2- 172.21.15.32:0/1542759399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e6c0cefe0 0x7f4e6c157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 >> 172.21.15.32:0/1542759399 conn(0x7f4e6c09bb50 msgr2=0x7f4e6c1554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 shutdown_connections 2023-10-16T08:18:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.008+0000 7f4e70cad640 1 -- 172.21.15.32:0/1542759399 wait complete. 2023-10-16T08:18:54.035 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T08:18:54.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:18:54.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 Processor -- start 2023-10-16T08:18:54.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 -- start start 2023-10-16T08:18:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6690070c40 0x7f669015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f66900b4020 0x7f66900b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f669015c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669015b3b0 con 0x7f6690070c40 2023-10-16T08:18:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900b0a70 con 0x7f66900b4020 2023-10-16T08:18:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6694e46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669015be60 con 0x7f66900b0620 2023-10-16T08:18:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6690070c40 0x7f669015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6690070c40 0x7f669015ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43552/0 (socket says 172.21.15.32:43552) 2023-10-16T08:18:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 -- 172.21.15.32:0/794068434 learned_addr learned my addr 172.21.15.32:0/794068434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6687fff640 1 --2- 172.21.15.32:0/794068434 >> v2:172.21.15.186:3300/0 conn(0x7f66900b4020 0x7f66900b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668ed9d640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f669015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 -- 172.21.15.32:0/794068434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 msgr2=0x7f669015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f669015c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 -- 172.21.15.32:0/794068434 >> v2:172.21.15.186:3300/0 conn(0x7f66900b4020 msgr2=0x7f66900b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 --2- 172.21.15.32:0/794068434 >> v2:172.21.15.186:3300/0 conn(0x7f66900b4020 0x7f66900b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 -- 172.21.15.32:0/794068434 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669015c1f0 con 0x7f6690070c40 2023-10-16T08:18:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f6687fff640 1 --2- 172.21.15.32:0/794068434 >> v2:172.21.15.186:3300/0 conn(0x7f66900b4020 0x7f66900b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668ed9d640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f669015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:54.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.176+0000 7f668e59c640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6690070c40 0x7f669015ae70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66900cc1e0 tx=0x7f668001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd89737fdfd956bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f668dd9b640 1 -- 172.21.15.32:0/794068434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66800a7070 con 0x7f6690070c40 2023-10-16T08:18:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f668dd9b640 1 -- 172.21.15.32:0/794068434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66800ab430 con 0x7f6690070c40 2023-10-16T08:18:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f668dd9b640 1 -- 172.21.15.32:0/794068434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66800ab7c0 con 0x7f6690070c40 2023-10-16T08:18:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- 172.21.15.32:0/794068434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6690070c40 msgr2=0x7f669015ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6690070c40 0x7f669015ae70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66900cc1e0 tx=0x7f668001c470 comp rx=0 tx=0).stop 2023-10-16T08:18:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- 172.21.15.32:0/794068434 shutdown_connections 2023-10-16T08:18:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- 172.21.15.32:0/794068434 >> v2:172.21.15.186:3300/0 conn(0x7f66900b4020 0x7f66900b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f669015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- 172.21.15.32:0/794068434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6690070c40 0x7f669015ae70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- 172.21.15.32:0/794068434 >> 172.21.15.32:0/794068434 conn(0x7f669006c190 msgr2=0x7f669006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- 172.21.15.32:0/794068434 shutdown_connections 2023-10-16T08:18:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- 172.21.15.32:0/794068434 wait complete. 2023-10-16T08:18:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 Processor -- start 2023-10-16T08:18:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- start start 2023-10-16T08:18:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6690070c40 0x7f66900c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f6690150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 0x7f66900c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690150c70 con 0x7f66900b4020 2023-10-16T08:18:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900c2ed0 con 0x7f6690070c40 2023-10-16T08:18:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f6694e46640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900c22c0 con 0x7f66900b0620 2023-10-16T08:18:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f668ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 0x7f66900c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.180+0000 7f668e59c640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6690070c40 0x7f66900c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6687fff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f6690150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 0x7f66900c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43554/0 (socket says 172.21.15.32:43554) 2023-10-16T08:18:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 -- 172.21.15.32:0/2398798466 learned_addr learned my addr 172.21.15.32:0/2398798466 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 -- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 msgr2=0x7f6690150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f6690150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 -- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6690070c40 msgr2=0x7f66900c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6690070c40 0x7f66900c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 -- 172.21.15.32:0/2398798466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800094d0 con 0x7f66900b4020 2023-10-16T08:18:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f668ed9d640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 0x7f66900c5730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f668800ce40 tx=0x7f668800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800d840 con 0x7f66900b4020 2023-10-16T08:18:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800d9a0 con 0x7f66900b4020 2023-10-16T08:18:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800dd50 con 0x7f66900b4020 2023-10-16T08:18:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900c25a0 con 0x7f66900b4020 2023-10-16T08:18:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900c2b60 con 0x7f66900b4020 2023-10-16T08:18:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900c3080 con 0x7f66900b4020 2023-10-16T08:18:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f668801e3b0 con 0x7f66900b4020 2023-10-16T08:18:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6686ffd640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66700664d0 0x7f6670068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.184+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f66880848c0 con 0x7f66900b4020 2023-10-16T08:18:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.188+0000 7f668e59c640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66700664d0 0x7f6670068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.188+0000 7f668e59c640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66700664d0 0x7f6670068990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66800af000 tx=0x7f6680008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.188+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f66900c3080 con 0x7f66900b4020 2023-10-16T08:18:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.344+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f669015bb80 con 0x7f66700664d0 2023-10-16T08:18:54.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6686ffd640 1 -- 172.21.15.32:0/2398798466 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289477 (secure 0 0 0) 0x7f66800bc090 con 0x7f66700664d0 2023-10-16T08:18:54.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:54.365 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:18:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66700664d0 msgr2=0x7f6670068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66700664d0 0x7f6670068990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66800af000 tx=0x7f6680008040 comp rx=0 tx=0).stop 2023-10-16T08:18:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 msgr2=0x7f66900c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 0x7f66900c5730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f668800ce40 tx=0x7f668800a710 comp rx=0 tx=0).stop 2023-10-16T08:18:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 shutdown_connections 2023-10-16T08:18:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66900b4020 0x7f66900c5730 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f66700664d0 0x7f6670068990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f66900b0620 0x7f6690150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 --2- 172.21.15.32:0/2398798466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6690070c40 0x7f66900c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 >> 172.21.15.32:0/2398798466 conn(0x7f669006c190 msgr2=0x7f6690153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 shutdown_connections 2023-10-16T08:18:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.356+0000 7f6694e46640 1 -- 172.21.15.32:0/2398798466 wait complete. 2023-10-16T08:18:54.384 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-10-16T08:18:54.050670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239108,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735612,"statfs":{"total":773094113280,"available":772849266688,"internally_reserved":0,"allocated":6430720,"data_stored":3497148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.806592"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036516+0000","last_change":"2023-10-16T08:18:50.293165+0000","last_active":"2023-10-16T08:18:52.036516+0000","last_peered":"2023-10-16T08:18:52.036516+0000","last_clean":"2023-10-16T08:18:52.036516+0000","last_became_active":"2023-10-16T08:18:50.292843+0000","last_became_peered":"2023-10-16T08:18:50.292843+0000","last_unstale":"2023-10-16T08:18:52.036516+0000","last_undegraded":"2023-10-16T08:18:52.036516+0000","last_fullsized":"2023-10-16T08:18:52.036516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:53.897776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114910+0000","last_change":"2023-10-16T08:18:50.290413+0000","last_active":"2023-10-16T08:18:52.114910+0000","last_peered":"2023-10-16T08:18:52.114910+0000","last_clean":"2023-10-16T08:18:52.114910+0000","last_became_active":"2023-10-16T08:18:50.290261+0000","last_became_peered":"2023-10-16T08:18:50.290261+0000","last_unstale":"2023-10-16T08:18:52.114910+0000","last_undegraded":"2023-10-16T08:18:52.114910+0000","last_fullsized":"2023-10-16T08:18:52.114910+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:43:35.553681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459572+0000","last_change":"2023-10-16T08:18:50.284610+0000","last_active":"2023-10-16T08:18:52.459572+0000","last_peered":"2023-10-16T08:18:52.459572+0000","last_clean":"2023-10-16T08:18:52.459572+0000","last_became_active":"2023-10-16T08:18:50.284331+0000","last_became_peered":"2023-10-16T08:18:50.284331+0000","last_unstale":"2023-10-16T08:18:52.459572+0000","last_undegraded":"2023-10-16T08:18:52.459572+0000","last_fullsized":"2023-10-16T08:18:52.459572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:38.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261764+0000","last_change":"2023-10-16T08:18:50.278407+0000","last_active":"2023-10-16T08:18:51.261764+0000","last_peered":"2023-10-16T08:18:51.261764+0000","last_clean":"2023-10-16T08:18:51.261764+0000","last_became_active":"2023-10-16T08:18:50.278143+0000","last_became_peered":"2023-10-16T08:18:50.278143+0000","last_unstale":"2023-10-16T08:18:51.261764+0000","last_undegraded":"2023-10-16T08:18:51.261764+0000","last_fullsized":"2023-10-16T08:18:51.261764+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:08.609731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036332+0000","last_change":"2023-10-16T08:18:50.279422+0000","last_active":"2023-10-16T08:18:52.036332+0000","last_peered":"2023-10-16T08:18:52.036332+0000","last_clean":"2023-10-16T08:18:52.036332+0000","last_became_active":"2023-10-16T08:18:50.279185+0000","last_became_peered":"2023-10-16T08:18:50.279185+0000","last_unstale":"2023-10-16T08:18:52.036332+0000","last_undegraded":"2023-10-16T08:18:52.036332+0000","last_fullsized":"2023-10-16T08:18:52.036332+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:32.567469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458867+0000","last_change":"2023-10-16T08:18:50.272152+0000","last_active":"2023-10-16T08:18:52.458867+0000","last_peered":"2023-10-16T08:18:52.458867+0000","last_clean":"2023-10-16T08:18:52.458867+0000","last_became_active":"2023-10-16T08:18:50.270371+0000","last_became_peered":"2023-10-16T08:18:50.270371+0000","last_unstale":"2023-10-16T08:18:52.458867+0000","last_undegraded":"2023-10-16T08:18:52.458867+0000","last_fullsized":"2023-10-16T08:18:52.458867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:38.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653438+0000","last_change":"2023-10-16T08:18:50.284250+0000","last_active":"2023-10-16T08:18:51.653438+0000","last_peered":"2023-10-16T08:18:51.653438+0000","last_clean":"2023-10-16T08:18:51.653438+0000","last_became_active":"2023-10-16T08:18:50.283263+0000","last_became_peered":"2023-10-16T08:18:50.283263+0000","last_unstale":"2023-10-16T08:18:51.653438+0000","last_undegraded":"2023-10-16T08:18:51.653438+0000","last_fullsized":"2023-10-16T08:18:51.653438+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:46:40.534100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114412+0000","last_change":"2023-10-16T08:18:50.275182+0000","last_active":"2023-10-16T08:18:52.114412+0000","last_peered":"2023-10-16T08:18:52.114412+0000","last_clean":"2023-10-16T08:18:52.114412+0000","last_became_active":"2023-10-16T08:18:50.274932+0000","last_became_peered":"2023-10-16T08:18:50.274932+0000","last_unstale":"2023-10-16T08:18:52.114412+0000","last_undegraded":"2023-10-16T08:18:52.114412+0000","last_fullsized":"2023-10-16T08:18:52.114412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:30.268242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653858+0000","last_change":"2023-10-16T08:18:50.285316+0000","last_active":"2023-10-16T08:18:51.653858+0000","last_peered":"2023-10-16T08:18:51.653858+0000","last_clean":"2023-10-16T08:18:51.653858+0000","last_became_active":"2023-10-16T08:18:50.285108+0000","last_became_peered":"2023-10-16T08:18:50.285108+0000","last_unstale":"2023-10-16T08:18:51.653858+0000","last_undegraded":"2023-10-16T08:18:51.653858+0000","last_fullsized":"2023-10-16T08:18:51.653858+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:45.628065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259995+0000","last_change":"2023-10-16T08:18:50.277091+0000","last_active":"2023-10-16T08:18:51.259995+0000","last_peered":"2023-10-16T08:18:51.259995+0000","last_clean":"2023-10-16T08:18:51.259995+0000","last_became_active":"2023-10-16T08:18:50.276777+0000","last_became_peered":"2023-10-16T08:18:50.276777+0000","last_unstale":"2023-10-16T08:18:51.259995+0000","last_undegraded":"2023-10-16T08:18:51.259995+0000","last_fullsized":"2023-10-16T08:18:51.259995+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:17.724968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259966+0000","last_change":"2023-10-16T08:18:50.290128+0000","last_active":"2023-10-16T08:18:51.259966+0000","last_peered":"2023-10-16T08:18:51.259966+0000","last_clean":"2023-10-16T08:18:51.259966+0000","last_became_active":"2023-10-16T08:18:50.289788+0000","last_became_peered":"2023-10-16T08:18:50.289788+0000","last_unstale":"2023-10-16T08:18:51.259966+0000","last_undegraded":"2023-10-16T08:18:51.259966+0000","last_fullsized":"2023-10-16T08:18:51.259966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:28.307736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261846+0000","last_change":"2023-10-16T08:18:50.278051+0000","last_active":"2023-10-16T08:18:51.261846+0000","last_peered":"2023-10-16T08:18:51.261846+0000","last_clean":"2023-10-16T08:18:51.261846+0000","last_became_active":"2023-10-16T08:18:50.277791+0000","last_became_peered":"2023-10-16T08:18:50.277791+0000","last_unstale":"2023-10-16T08:18:51.261846+0000","last_undegraded":"2023-10-16T08:18:51.261846+0000","last_fullsized":"2023-10-16T08:18:51.261846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:38.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.262011+0000","last_change":"2023-10-16T08:18:50.274455+0000","last_active":"2023-10-16T08:18:51.262011+0000","last_peered":"2023-10-16T08:18:51.262011+0000","last_clean":"2023-10-16T08:18:51.262011+0000","last_became_active":"2023-10-16T08:18:50.274172+0000","last_became_peered":"2023-10-16T08:18:50.274172+0000","last_unstale":"2023-10-16T08:18:51.262011+0000","last_undegraded":"2023-10-16T08:18:51.262011+0000","last_fullsized":"2023-10-16T08:18:51.262011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:53.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653632+0000","last_change":"2023-10-16T08:18:50.283561+0000","last_active":"2023-10-16T08:18:51.653632+0000","last_peered":"2023-10-16T08:18:51.653632+0000","last_clean":"2023-10-16T08:18:51.653632+0000","last_became_active":"2023-10-16T08:18:50.283289+0000","last_became_peered":"2023-10-16T08:18:50.283289+0000","last_unstale":"2023-10-16T08:18:51.653632+0000","last_undegraded":"2023-10-16T08:18:51.653632+0000","last_fullsized":"2023-10-16T08:18:51.653632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:58.648055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653560+0000","last_change":"2023-10-16T08:18:50.275300+0000","last_active":"2023-10-16T08:18:51.653560+0000","last_peered":"2023-10-16T08:18:51.653560+0000","last_clean":"2023-10-16T08:18:51.653560+0000","last_became_active":"2023-10-16T08:18:50.274924+0000","last_became_peered":"2023-10-16T08:18:50.274924+0000","last_unstale":"2023-10-16T08:18:51.653560+0000","last_undegraded":"2023-10-16T08:18:51.653560+0000","last_fullsized":"2023-10-16T08:18:51.653560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:48.800013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458798+0000","last_change":"2023-10-16T08:18:50.268536+0000","last_active":"2023-10-16T08:18:52.458798+0000","last_peered":"2023-10-16T08:18:52.458798+0000","last_clean":"2023-10-16T08:18:52.458798+0000","last_became_active":"2023-10-16T08:18:50.268265+0000","last_became_peered":"2023-10-16T08:18:50.268265+0000","last_unstale":"2023-10-16T08:18:52.458798+0000","last_undegraded":"2023-10-16T08:18:52.458798+0000","last_fullsized":"2023-10-16T08:18:52.458798+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:13.623970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.115127+0000","last_change":"2023-10-16T08:18:50.276117+0000","last_active":"2023-10-16T08:18:52.115127+0000","last_peered":"2023-10-16T08:18:52.115127+0000","last_clean":"2023-10-16T08:18:52.115127+0000","last_became_active":"2023-10-16T08:18:50.275943+0000","last_became_peered":"2023-10-16T08:18:50.275943+0000","last_unstale":"2023-10-16T08:18:52.115127+0000","last_undegraded":"2023-10-16T08:18:52.115127+0000","last_fullsized":"2023-10-16T08:18:52.115127+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:18.861708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459714+0000","last_change":"2023-10-16T08:18:50.285032+0000","last_active":"2023-10-16T08:18:52.459714+0000","last_peered":"2023-10-16T08:18:52.459714+0000","last_clean":"2023-10-16T08:18:52.459714+0000","last_became_active":"2023-10-16T08:18:50.284742+0000","last_became_peered":"2023-10-16T08:18:50.284742+0000","last_unstale":"2023-10-16T08:18:52.459714+0000","last_undegraded":"2023-10-16T08:18:52.459714+0000","last_fullsized":"2023-10-16T08:18:52.459714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:36.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036325+0000","last_change":"2023-10-16T08:18:50.303134+0000","last_active":"2023-10-16T08:18:52.036325+0000","last_peered":"2023-10-16T08:18:52.036325+0000","last_clean":"2023-10-16T08:18:52.036325+0000","last_became_active":"2023-10-16T08:18:50.302843+0000","last_became_peered":"2023-10-16T08:18:50.302843+0000","last_unstale":"2023-10-16T08:18:52.036325+0000","last_undegraded":"2023-10-16T08:18:52.036325+0000","last_fullsized":"2023-10-16T08:18:52.036325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:33.173663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114852+0000","last_change":"2023-10-16T08:18:50.280459+0000","last_active":"2023-10-16T08:18:52.114852+0000","last_peered":"2023-10-16T08:18:52.114852+0000","last_clean":"2023-10-16T08:18:52.114852+0000","last_became_active":"2023-10-16T08:18:50.280283+0000","last_became_peered":"2023-10-16T08:18:50.280283+0000","last_unstale":"2023-10-16T08:18:52.114852+0000","last_undegraded":"2023-10-16T08:18:52.114852+0000","last_fullsized":"2023-10-16T08:18:52.114852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:58.958197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459262+0000","last_change":"2023-10-16T08:18:50.285379+0000","last_active":"2023-10-16T08:18:52.459262+0000","last_peered":"2023-10-16T08:18:52.459262+0000","last_clean":"2023-10-16T08:18:52.459262+0000","last_became_active":"2023-10-16T08:18:50.285090+0000","last_became_peered":"2023-10-16T08:18:50.285090+0000","last_unstale":"2023-10-16T08:18:52.459262+0000","last_undegraded":"2023-10-16T08:18:52.459262+0000","last_fullsized":"2023-10-16T08:18:52.459262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:09.144644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114483+0000","last_change":"2023-10-16T08:18:50.291868+0000","last_active":"2023-10-16T08:18:52.114483+0000","last_peered":"2023-10-16T08:18:52.114483+0000","last_clean":"2023-10-16T08:18:52.114483+0000","last_became_active":"2023-10-16T08:18:50.286778+0000","last_became_peered":"2023-10-16T08:18:50.286778+0000","last_unstale":"2023-10-16T08:18:52.114483+0000","last_undegraded":"2023-10-16T08:18:52.114483+0000","last_fullsized":"2023-10-16T08:18:52.114483+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:11:53.180528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459229+0000","last_change":"2023-10-16T08:18:50.287810+0000","last_active":"2023-10-16T08:18:52.459229+0000","last_peered":"2023-10-16T08:18:52.459229+0000","last_clean":"2023-10-16T08:18:52.459229+0000","last_became_active":"2023-10-16T08:18:50.287378+0000","last_became_peered":"2023-10-16T08:18:50.287378+0000","last_unstale":"2023-10-16T08:18:52.459229+0000","last_undegraded":"2023-10-16T08:18:52.459229+0000","last_fullsized":"2023-10-16T08:18:52.459229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:49.807775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261185+0000","last_change":"2023-10-16T08:18:50.273131+0000","last_active":"2023-10-16T08:18:51.261185+0000","last_peered":"2023-10-16T08:18:51.261185+0000","last_clean":"2023-10-16T08:18:51.261185+0000","last_became_active":"2023-10-16T08:18:50.272572+0000","last_became_peered":"2023-10-16T08:18:50.272572+0000","last_unstale":"2023-10-16T08:18:51.261185+0000","last_undegraded":"2023-10-16T08:18:51.261185+0000","last_fullsized":"2023-10-16T08:18:51.261185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:26:34.317445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036575+0000","last_change":"2023-10-16T08:18:50.299812+0000","last_active":"2023-10-16T08:18:52.036575+0000","last_peered":"2023-10-16T08:18:52.036575+0000","last_clean":"2023-10-16T08:18:52.036575+0000","last_became_active":"2023-10-16T08:18:50.299497+0000","last_became_peered":"2023-10-16T08:18:50.299497+0000","last_unstale":"2023-10-16T08:18:52.036575+0000","last_undegraded":"2023-10-16T08:18:52.036575+0000","last_fullsized":"2023-10-16T08:18:52.036575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:57.670386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261444+0000","last_change":"2023-10-16T08:18:50.271399+0000","last_active":"2023-10-16T08:18:51.261444+0000","last_peered":"2023-10-16T08:18:51.261444+0000","last_clean":"2023-10-16T08:18:51.261444+0000","last_became_active":"2023-10-16T08:18:50.271125+0000","last_became_peered":"2023-10-16T08:18:50.271125+0000","last_unstale":"2023-10-16T08:18:51.261444+0000","last_undegraded":"2023-10-16T08:18:51.261444+0000","last_fullsized":"2023-10-16T08:18:51.261444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:13.623970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459198+0000","last_change":"2023-10-16T08:18:50.272337+0000","last_active":"2023-10-16T08:18:52.459198+0000","last_peered":"2023-10-16T08:18:52.459198+0000","last_clean":"2023-10-16T08:18:52.459198+0000","last_became_active":"2023-10-16T08:18:50.271938+0000","last_became_peered":"2023-10-16T08:18:50.271938+0000","last_unstale":"2023-10-16T08:18:52.459198+0000","last_undegraded":"2023-10-16T08:18:52.459198+0000","last_fullsized":"2023-10-16T08:18:52.459198+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:36.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036384+0000","last_change":"2023-10-16T08:18:50.279472+0000","last_active":"2023-10-16T08:18:52.036384+0000","last_peered":"2023-10-16T08:18:52.036384+0000","last_clean":"2023-10-16T08:18:52.036384+0000","last_became_active":"2023-10-16T08:18:50.279171+0000","last_became_peered":"2023-10-16T08:18:50.279171+0000","last_unstale":"2023-10-16T08:18:52.036384+0000","last_undegraded":"2023-10-16T08:18:52.036384+0000","last_fullsized":"2023-10-16T08:18:52.036384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:13.057328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114847+0000","last_change":"2023-10-16T08:18:50.275373+0000","last_active":"2023-10-16T08:18:52.114847+0000","last_peered":"2023-10-16T08:18:52.114847+0000","last_clean":"2023-10-16T08:18:52.114847+0000","last_became_active":"2023-10-16T08:18:50.275101+0000","last_became_peered":"2023-10-16T08:18:50.275101+0000","last_unstale":"2023-10-16T08:18:52.114847+0000","last_undegraded":"2023-10-16T08:18:52.114847+0000","last_fullsized":"2023-10-16T08:18:52.114847+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:34.987039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459323+0000","last_change":"2023-10-16T08:18:50.282594+0000","last_active":"2023-10-16T08:18:52.459323+0000","last_peered":"2023-10-16T08:18:52.459323+0000","last_clean":"2023-10-16T08:18:52.459323+0000","last_became_active":"2023-10-16T08:18:50.282378+0000","last_became_peered":"2023-10-16T08:18:50.282378+0000","last_unstale":"2023-10-16T08:18:52.459323+0000","last_undegraded":"2023-10-16T08:18:52.459323+0000","last_fullsized":"2023-10-16T08:18:52.459323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:00.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036269+0000","last_change":"2023-10-16T08:18:50.297269+0000","last_active":"2023-10-16T08:18:52.036269+0000","last_peered":"2023-10-16T08:18:52.036269+0000","last_clean":"2023-10-16T08:18:52.036269+0000","last_became_active":"2023-10-16T08:18:50.296997+0000","last_became_peered":"2023-10-16T08:18:50.296997+0000","last_unstale":"2023-10-16T08:18:52.036269+0000","last_undegraded":"2023-10-16T08:18:52.036269+0000","last_fullsized":"2023-10-16T08:18:52.036269+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:58.116685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458867+0000","last_change":"2023-10-16T08:18:50.278627+0000","last_active":"2023-10-16T08:18:52.458867+0000","last_peered":"2023-10-16T08:18:52.458867+0000","last_clean":"2023-10-16T08:18:52.458867+0000","last_became_active":"2023-10-16T08:18:50.278340+0000","last_became_peered":"2023-10-16T08:18:50.278340+0000","last_unstale":"2023-10-16T08:18:52.458867+0000","last_undegraded":"2023-10-16T08:18:52.458867+0000","last_fullsized":"2023-10-16T08:18:52.458867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:08.609731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459163+0000","last_change":"2023-10-16T08:18:50.280362+0000","last_active":"2023-10-16T08:18:52.459163+0000","last_peered":"2023-10-16T08:18:52.459163+0000","last_clean":"2023-10-16T08:18:52.459163+0000","last_became_active":"2023-10-16T08:18:50.280096+0000","last_became_peered":"2023-10-16T08:18:50.280096+0000","last_unstale":"2023-10-16T08:18:52.459163+0000","last_undegraded":"2023-10-16T08:18:52.459163+0000","last_fullsized":"2023-10-16T08:18:52.459163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:18.275313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.301561+0000","last_change":"2023-10-16T08:18:49.280394+0000","last_active":"2023-10-16T08:18:50.301561+0000","last_peered":"2023-10-16T08:18:50.301561+0000","last_clean":"2023-10-16T08:18:50.301561+0000","last_became_active":"2023-10-16T08:18:49.280087+0000","last_became_peered":"2023-10-16T08:18:49.280087+0000","last_unstale":"2023-10-16T08:18:50.301561+0000","last_undegraded":"2023-10-16T08:18:50.301561+0000","last_fullsized":"2023-10-16T08:18:50.301561+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:27.300826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.260052+0000","last_change":"2023-10-16T08:18:50.276206+0000","last_active":"2023-10-16T08:18:51.260052+0000","last_peered":"2023-10-16T08:18:51.260052+0000","last_clean":"2023-10-16T08:18:51.260052+0000","last_became_active":"2023-10-16T08:18:50.275914+0000","last_became_peered":"2023-10-16T08:18:50.275914+0000","last_unstale":"2023-10-16T08:18:51.260052+0000","last_undegraded":"2023-10-16T08:18:51.260052+0000","last_fullsized":"2023-10-16T08:18:51.260052+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:19:06.070977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.297199+0000","last_change":"2023-10-16T08:18:49.283430+0000","last_active":"2023-10-16T08:18:50.297199+0000","last_peered":"2023-10-16T08:18:50.297199+0000","last_clean":"2023-10-16T08:18:50.297199+0000","last_became_active":"2023-10-16T08:18:49.283116+0000","last_became_peered":"2023-10-16T08:18:49.283116+0000","last_unstale":"2023-10-16T08:18:50.297199+0000","last_undegraded":"2023-10-16T08:18:50.297199+0000","last_fullsized":"2023-10-16T08:18:50.297199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:15.786640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459338+0000","last_change":"2023-10-16T08:18:50.271957+0000","last_active":"2023-10-16T08:18:52.459338+0000","last_peered":"2023-10-16T08:18:52.459338+0000","last_clean":"2023-10-16T08:18:52.459338+0000","last_became_active":"2023-10-16T08:18:50.269540+0000","last_became_peered":"2023-10-16T08:18:50.269540+0000","last_unstale":"2023-10-16T08:18:52.459338+0000","last_undegraded":"2023-10-16T08:18:52.459338+0000","last_fullsized":"2023-10-16T08:18:52.459338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:56:07.404247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.302520+0000","last_change":"2023-10-16T08:18:49.275196+0000","last_active":"2023-10-16T08:18:50.302520+0000","last_peered":"2023-10-16T08:18:50.302520+0000","last_clean":"2023-10-16T08:18:50.302520+0000","last_became_active":"2023-10-16T08:18:49.274931+0000","last_became_peered":"2023-10-16T08:18:49.274931+0000","last_unstale":"2023-10-16T08:18:50.302520+0000","last_undegraded":"2023-10-16T08:18:50.302520+0000","last_fullsized":"2023-10-16T08:18:50.302520+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:59.173567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459295+0000","last_change":"2023-10-16T08:18:50.282434+0000","last_active":"2023-10-16T08:18:52.459295+0000","last_peered":"2023-10-16T08:18:52.459295+0000","last_clean":"2023-10-16T08:18:52.459295+0000","last_became_active":"2023-10-16T08:18:50.281408+0000","last_became_peered":"2023-10-16T08:18:50.281408+0000","last_unstale":"2023-10-16T08:18:52.459295+0000","last_undegraded":"2023-10-16T08:18:52.459295+0000","last_fullsized":"2023-10-16T08:18:52.459295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:54.136989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296203+0000","last_change":"2023-10-16T08:18:49.279763+0000","last_active":"2023-10-16T08:18:50.296203+0000","last_peered":"2023-10-16T08:18:50.296203+0000","last_clean":"2023-10-16T08:18:50.296203+0000","last_became_active":"2023-10-16T08:18:49.279483+0000","last_became_peered":"2023-10-16T08:18:49.279483+0000","last_unstale":"2023-10-16T08:18:50.296203+0000","last_undegraded":"2023-10-16T08:18:50.296203+0000","last_fullsized":"2023-10-16T08:18:50.296203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:39.925292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114477+0000","last_change":"2023-10-16T08:18:50.286758+0000","last_active":"2023-10-16T08:18:52.114477+0000","last_peered":"2023-10-16T08:18:52.114477+0000","last_clean":"2023-10-16T08:18:52.114477+0000","last_became_active":"2023-10-16T08:18:50.286487+0000","last_became_peered":"2023-10-16T08:18:50.286487+0000","last_unstale":"2023-10-16T08:18:52.114477+0000","last_undegraded":"2023-10-16T08:18:52.114477+0000","last_fullsized":"2023-10-16T08:18:52.114477+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:26.466105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.295708+0000","last_change":"2023-10-16T08:18:49.270849+0000","last_active":"2023-10-16T08:18:50.295708+0000","last_peered":"2023-10-16T08:18:50.295708+0000","last_clean":"2023-10-16T08:18:50.295708+0000","last_became_active":"2023-10-16T08:18:49.270148+0000","last_became_peered":"2023-10-16T08:18:49.270148+0000","last_unstale":"2023-10-16T08:18:50.295708+0000","last_undegraded":"2023-10-16T08:18:50.295708+0000","last_fullsized":"2023-10-16T08:18:50.295708+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:08.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114643+0000","last_change":"2023-10-16T08:18:50.272082+0000","last_active":"2023-10-16T08:18:52.114643+0000","last_peered":"2023-10-16T08:18:52.114643+0000","last_clean":"2023-10-16T08:18:52.114643+0000","last_became_active":"2023-10-16T08:18:50.271809+0000","last_became_peered":"2023-10-16T08:18:50.271809+0000","last_unstale":"2023-10-16T08:18:52.114643+0000","last_undegraded":"2023-10-16T08:18:52.114643+0000","last_fullsized":"2023-10-16T08:18:52.114643+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:22.987216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296324+0000","last_change":"2023-10-16T08:18:49.272914+0000","last_active":"2023-10-16T08:18:50.296324+0000","last_peered":"2023-10-16T08:18:50.296324+0000","last_clean":"2023-10-16T08:18:50.296324+0000","last_became_active":"2023-10-16T08:18:49.272652+0000","last_became_peered":"2023-10-16T08:18:49.272652+0000","last_unstale":"2023-10-16T08:18:50.296324+0000","last_undegraded":"2023-10-16T08:18:50.296324+0000","last_fullsized":"2023-10-16T08:18:50.296324+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:16.718057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261334+0000","last_change":"2023-10-16T08:18:50.275700+0000","last_active":"2023-10-16T08:18:51.261334+0000","last_peered":"2023-10-16T08:18:51.261334+0000","last_clean":"2023-10-16T08:18:51.261334+0000","last_became_active":"2023-10-16T08:18:50.275442+0000","last_became_peered":"2023-10-16T08:18:50.275442+0000","last_unstale":"2023-10-16T08:18:51.261334+0000","last_undegraded":"2023-10-16T08:18:51.261334+0000","last_fullsized":"2023-10-16T08:18:51.261334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:09.092916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.262597+0000","last_change":"2023-10-16T08:18:49.279275+0000","last_active":"2023-10-16T08:18:50.262597+0000","last_peered":"2023-10-16T08:18:50.262597+0000","last_clean":"2023-10-16T08:18:50.262597+0000","last_became_active":"2023-10-16T08:18:49.278699+0000","last_became_peered":"2023-10-16T08:18:49.278699+0000","last_unstale":"2023-10-16T08:18:50.262597+0000","last_undegraded":"2023-10-16T08:18:50.262597+0000","last_fullsized":"2023-10-16T08:18:50.262597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:24.205203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114572+0000","last_change":"2023-10-16T08:18:50.277601+0000","last_active":"2023-10-16T08:18:52.114572+0000","last_peered":"2023-10-16T08:18:52.114572+0000","last_clean":"2023-10-16T08:18:52.114572+0000","last_became_active":"2023-10-16T08:18:50.277357+0000","last_became_peered":"2023-10-16T08:18:50.277357+0000","last_unstale":"2023-10-16T08:18:52.114572+0000","last_undegraded":"2023-10-16T08:18:52.114572+0000","last_fullsized":"2023-10-16T08:18:52.114572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:21.664383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.265797+0000","last_change":"2023-10-16T08:18:49.278010+0000","last_active":"2023-10-16T08:18:50.265797+0000","last_peered":"2023-10-16T08:18:50.265797+0000","last_clean":"2023-10-16T08:18:50.265797+0000","last_became_active":"2023-10-16T08:18:49.277690+0000","last_became_peered":"2023-10-16T08:18:49.277690+0000","last_unstale":"2023-10-16T08:18:50.265797+0000","last_undegraded":"2023-10-16T08:18:50.265797+0000","last_fullsized":"2023-10-16T08:18:50.265797+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:10:22.077040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653865+0000","last_change":"2023-10-16T08:18:50.280582+0000","last_active":"2023-10-16T08:18:51.653865+0000","last_peered":"2023-10-16T08:18:51.653865+0000","last_clean":"2023-10-16T08:18:51.653865+0000","last_became_active":"2023-10-16T08:18:50.280318+0000","last_became_peered":"2023-10-16T08:18:50.280318+0000","last_unstale":"2023-10-16T08:18:51.653865+0000","last_undegraded":"2023-10-16T08:18:51.653865+0000","last_fullsized":"2023-10-16T08:18:51.653865+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:28.307736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.273847+0000","last_change":"2023-10-16T08:18:49.282731+0000","last_active":"2023-10-16T08:18:50.273847+0000","last_peered":"2023-10-16T08:18:50.273847+0000","last_clean":"2023-10-16T08:18:50.273847+0000","last_became_active":"2023-10-16T08:18:49.282408+0000","last_became_peered":"2023-10-16T08:18:49.282408+0000","last_unstale":"2023-10-16T08:18:50.273847+0000","last_undegraded":"2023-10-16T08:18:50.273847+0000","last_fullsized":"2023-10-16T08:18:50.273847+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.847211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459812+0000","last_change":"2023-10-16T08:18:50.287101+0000","last_active":"2023-10-16T08:18:52.459812+0000","last_peered":"2023-10-16T08:18:52.459812+0000","last_clean":"2023-10-16T08:18:52.459812+0000","last_became_active":"2023-10-16T08:18:50.286807+0000","last_became_peered":"2023-10-16T08:18:50.286807+0000","last_unstale":"2023-10-16T08:18:52.459812+0000","last_undegraded":"2023-10-16T08:18:52.459812+0000","last_fullsized":"2023-10-16T08:18:52.459812+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:10.454139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.267966+0000","last_change":"2023-10-16T08:18:49.276851+0000","last_active":"2023-10-16T08:18:50.267966+0000","last_peered":"2023-10-16T08:18:50.267966+0000","last_clean":"2023-10-16T08:18:50.267966+0000","last_became_active":"2023-10-16T08:18:49.276615+0000","last_became_peered":"2023-10-16T08:18:49.276615+0000","last_unstale":"2023-10-16T08:18:50.267966+0000","last_undegraded":"2023-10-16T08:18:50.267966+0000","last_fullsized":"2023-10-16T08:18:50.267966+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:29.702452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259756+0000","last_change":"2023-10-16T08:18:50.271193+0000","last_active":"2023-10-16T08:18:51.259756+0000","last_peered":"2023-10-16T08:18:51.259756+0000","last_clean":"2023-10-16T08:18:51.259756+0000","last_became_active":"2023-10-16T08:18:50.270850+0000","last_became_peered":"2023-10-16T08:18:50.270850+0000","last_unstale":"2023-10-16T08:18:51.259756+0000","last_undegraded":"2023-10-16T08:18:51.259756+0000","last_fullsized":"2023-10-16T08:18:51.259756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:45.628065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.299274+0000","last_change":"2023-10-16T08:18:49.283642+0000","last_active":"2023-10-16T08:18:50.299274+0000","last_peered":"2023-10-16T08:18:50.299274+0000","last_clean":"2023-10-16T08:18:50.299274+0000","last_became_active":"2023-10-16T08:18:49.283364+0000","last_became_peered":"2023-10-16T08:18:49.283364+0000","last_unstale":"2023-10-16T08:18:50.299274+0000","last_undegraded":"2023-10-16T08:18:50.299274+0000","last_fullsized":"2023-10-16T08:18:50.299274+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:39.925292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459598+0000","last_change":"2023-10-16T08:18:50.297192+0000","last_active":"2023-10-16T08:18:52.459598+0000","last_peered":"2023-10-16T08:18:52.459598+0000","last_clean":"2023-10-16T08:18:52.459598+0000","last_became_active":"2023-10-16T08:18:50.296934+0000","last_became_peered":"2023-10-16T08:18:50.296934+0000","last_unstale":"2023-10-16T08:18:52.459598+0000","last_undegraded":"2023-10-16T08:18:52.459598+0000","last_fullsized":"2023-10-16T08:18:52.459598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:18.275313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.302553+0000","last_change":"2023-10-16T08:18:49.276927+0000","last_active":"2023-10-16T08:18:50.302553+0000","last_peered":"2023-10-16T08:18:50.302553+0000","last_clean":"2023-10-16T08:18:50.302553+0000","last_became_active":"2023-10-16T08:18:49.276611+0000","last_became_peered":"2023-10-16T08:18:49.276611+0000","last_unstale":"2023-10-16T08:18:50.302553+0000","last_undegraded":"2023-10-16T08:18:50.302553+0000","last_fullsized":"2023-10-16T08:18:50.302553+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:51.038960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261904+0000","last_change":"2023-10-16T08:18:50.269002+0000","last_active":"2023-10-16T08:18:51.261904+0000","last_peered":"2023-10-16T08:18:51.261904+0000","last_clean":"2023-10-16T08:18:51.261904+0000","last_became_active":"2023-10-16T08:18:50.268635+0000","last_became_peered":"2023-10-16T08:18:50.268635+0000","last_unstale":"2023-10-16T08:18:51.261904+0000","last_undegraded":"2023-10-16T08:18:51.261904+0000","last_fullsized":"2023-10-16T08:18:51.261904+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:36.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.305941+0000","last_change":"2023-10-16T08:18:49.280831+0000","last_active":"2023-10-16T08:18:50.305941+0000","last_peered":"2023-10-16T08:18:50.305941+0000","last_clean":"2023-10-16T08:18:50.305941+0000","last_became_active":"2023-10-16T08:18:49.280588+0000","last_became_peered":"2023-10-16T08:18:49.280588+0000","last_unstale":"2023-10-16T08:18:50.305941+0000","last_undegraded":"2023-10-16T08:18:50.305941+0000","last_fullsized":"2023-10-16T08:18:50.305941+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:24:55.934837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459427+0000","last_change":"2023-10-16T08:18:50.281051+0000","last_active":"2023-10-16T08:18:52.459427+0000","last_peered":"2023-10-16T08:18:52.459427+0000","last_clean":"2023-10-16T08:18:52.459427+0000","last_became_active":"2023-10-16T08:18:50.280696+0000","last_became_peered":"2023-10-16T08:18:50.280696+0000","last_unstale":"2023-10-16T08:18:52.459427+0000","last_undegraded":"2023-10-16T08:18:52.459427+0000","last_fullsized":"2023-10-16T08:18:52.459427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:13.623970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.259952+0000","last_change":"2023-10-16T08:18:49.280310+0000","last_active":"2023-10-16T08:18:50.259952+0000","last_peered":"2023-10-16T08:18:50.259952+0000","last_clean":"2023-10-16T08:18:50.259952+0000","last_became_active":"2023-10-16T08:18:49.279440+0000","last_became_peered":"2023-10-16T08:18:49.279440+0000","last_unstale":"2023-10-16T08:18:50.259952+0000","last_undegraded":"2023-10-16T08:18:50.259952+0000","last_fullsized":"2023-10-16T08:18:50.259952+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:51.038960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653443+0000","last_change":"2023-10-16T08:18:50.277009+0000","last_active":"2023-10-16T08:18:51.653443+0000","last_peered":"2023-10-16T08:18:51.653443+0000","last_clean":"2023-10-16T08:18:51.653443+0000","last_became_active":"2023-10-16T08:18:50.276669+0000","last_became_peered":"2023-10-16T08:18:50.276669+0000","last_unstale":"2023-10-16T08:18:51.653443+0000","last_undegraded":"2023-10-16T08:18:51.653443+0000","last_fullsized":"2023-10-16T08:18:51.653443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:02.526716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.262764+0000","last_change":"2023-10-16T08:18:49.275328+0000","last_active":"2023-10-16T08:18:50.262764+0000","last_peered":"2023-10-16T08:18:50.262764+0000","last_clean":"2023-10-16T08:18:50.262764+0000","last_became_active":"2023-10-16T08:18:49.275006+0000","last_became_peered":"2023-10-16T08:18:49.275006+0000","last_unstale":"2023-10-16T08:18:50.262764+0000","last_undegraded":"2023-10-16T08:18:50.262764+0000","last_fullsized":"2023-10-16T08:18:50.262764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:48.800864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459445+0000","last_change":"2023-10-16T08:18:50.295996+0000","last_active":"2023-10-16T08:18:52.459445+0000","last_peered":"2023-10-16T08:18:52.459445+0000","last_clean":"2023-10-16T08:18:52.459445+0000","last_became_active":"2023-10-16T08:18:50.295617+0000","last_became_peered":"2023-10-16T08:18:50.295617+0000","last_unstale":"2023-10-16T08:18:52.459445+0000","last_undegraded":"2023-10-16T08:18:52.459445+0000","last_fullsized":"2023-10-16T08:18:52.459445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:13.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296704+0000","last_change":"2023-10-16T08:18:49.283459+0000","last_active":"2023-10-16T08:18:50.296704+0000","last_peered":"2023-10-16T08:18:50.296704+0000","last_clean":"2023-10-16T08:18:50.296704+0000","last_became_active":"2023-10-16T08:18:49.282767+0000","last_became_peered":"2023-10-16T08:18:49.282767+0000","last_unstale":"2023-10-16T08:18:50.296704+0000","last_undegraded":"2023-10-16T08:18:50.296704+0000","last_fullsized":"2023-10-16T08:18:50.296704+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:08.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459747+0000","last_change":"2023-10-16T08:18:50.282534+0000","last_active":"2023-10-16T08:18:52.459747+0000","last_peered":"2023-10-16T08:18:52.459747+0000","last_clean":"2023-10-16T08:18:52.459747+0000","last_became_active":"2023-10-16T08:18:50.282208+0000","last_became_peered":"2023-10-16T08:18:50.282208+0000","last_unstale":"2023-10-16T08:18:52.459747+0000","last_undegraded":"2023-10-16T08:18:52.459747+0000","last_fullsized":"2023-10-16T08:18:52.459747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:01.834834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.305988+0000","last_change":"2023-10-16T08:18:49.277089+0000","last_active":"2023-10-16T08:18:50.305988+0000","last_peered":"2023-10-16T08:18:50.305988+0000","last_clean":"2023-10-16T08:18:50.305988+0000","last_became_active":"2023-10-16T08:18:49.276783+0000","last_became_peered":"2023-10-16T08:18:49.276783+0000","last_unstale":"2023-10-16T08:18:50.305988+0000","last_undegraded":"2023-10-16T08:18:50.305988+0000","last_fullsized":"2023-10-16T08:18:50.305988+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:42.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261872+0000","last_change":"2023-10-16T08:18:50.271171+0000","last_active":"2023-10-16T08:18:51.261872+0000","last_peered":"2023-10-16T08:18:51.261872+0000","last_clean":"2023-10-16T08:18:51.261872+0000","last_became_active":"2023-10-16T08:18:50.270887+0000","last_became_peered":"2023-10-16T08:18:50.270887+0000","last_unstale":"2023-10-16T08:18:51.261872+0000","last_undegraded":"2023-10-16T08:18:51.261872+0000","last_fullsized":"2023-10-16T08:18:51.261872+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:18.275313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.305345+0000","last_change":"2023-10-16T08:18:49.279592+0000","last_active":"2023-10-16T08:18:50.305345+0000","last_peered":"2023-10-16T08:18:50.305345+0000","last_clean":"2023-10-16T08:18:50.305345+0000","last_became_active":"2023-10-16T08:18:49.279318+0000","last_became_peered":"2023-10-16T08:18:49.279318+0000","last_unstale":"2023-10-16T08:18:50.305345+0000","last_undegraded":"2023-10-16T08:18:50.305345+0000","last_fullsized":"2023-10-16T08:18:50.305345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:39.925292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114895+0000","last_change":"2023-10-16T08:18:50.270888+0000","last_active":"2023-10-16T08:18:52.114895+0000","last_peered":"2023-10-16T08:18:52.114895+0000","last_clean":"2023-10-16T08:18:52.114895+0000","last_became_active":"2023-10-16T08:18:50.270565+0000","last_became_peered":"2023-10-16T08:18:50.270565+0000","last_unstale":"2023-10-16T08:18:52.114895+0000","last_undegraded":"2023-10-16T08:18:52.114895+0000","last_fullsized":"2023-10-16T08:18:52.114895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:18:17.973276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.294474+0000","last_change":"2023-10-16T08:18:49.280302+0000","last_active":"2023-10-16T08:18:50.294474+0000","last_peered":"2023-10-16T08:18:50.294474+0000","last_clean":"2023-10-16T08:18:50.294474+0000","last_became_active":"2023-10-16T08:18:49.280011+0000","last_became_peered":"2023-10-16T08:18:49.280011+0000","last_unstale":"2023-10-16T08:18:50.294474+0000","last_undegraded":"2023-10-16T08:18:50.294474+0000","last_fullsized":"2023-10-16T08:18:50.294474+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:08.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459751+0000","last_change":"2023-10-16T08:18:50.297198+0000","last_active":"2023-10-16T08:18:52.459751+0000","last_peered":"2023-10-16T08:18:52.459751+0000","last_clean":"2023-10-16T08:18:52.459751+0000","last_became_active":"2023-10-16T08:18:50.296966+0000","last_became_peered":"2023-10-16T08:18:50.296966+0000","last_unstale":"2023-10-16T08:18:52.459751+0000","last_undegraded":"2023-10-16T08:18:52.459751+0000","last_fullsized":"2023-10-16T08:18:52.459751+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:54.136989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.261924+0000","last_change":"2023-10-16T08:18:49.275038+0000","last_active":"2023-10-16T08:18:50.261924+0000","last_peered":"2023-10-16T08:18:50.261924+0000","last_clean":"2023-10-16T08:18:50.261924+0000","last_became_active":"2023-10-16T08:18:49.274675+0000","last_became_peered":"2023-10-16T08:18:49.274675+0000","last_unstale":"2023-10-16T08:18:50.261924+0000","last_undegraded":"2023-10-16T08:18:50.261924+0000","last_fullsized":"2023-10-16T08:18:50.261924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.847211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.262036+0000","last_change":"2023-10-16T08:18:50.268973+0000","last_active":"2023-10-16T08:18:51.262036+0000","last_peered":"2023-10-16T08:18:51.262036+0000","last_clean":"2023-10-16T08:18:51.262036+0000","last_became_active":"2023-10-16T08:18:50.268631+0000","last_became_peered":"2023-10-16T08:18:50.268631+0000","last_unstale":"2023-10-16T08:18:51.262036+0000","last_undegraded":"2023-10-16T08:18:51.262036+0000","last_fullsized":"2023-10-16T08:18:51.262036+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:54.136989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296927+0000","last_change":"2023-10-16T08:18:49.283050+0000","last_active":"2023-10-16T08:18:50.296927+0000","last_peered":"2023-10-16T08:18:50.296927+0000","last_clean":"2023-10-16T08:18:50.296927+0000","last_became_active":"2023-10-16T08:18:49.282767+0000","last_became_peered":"2023-10-16T08:18:49.282767+0000","last_unstale":"2023-10-16T08:18:50.296927+0000","last_undegraded":"2023-10-16T08:18:50.296927+0000","last_fullsized":"2023-10-16T08:18:50.296927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:51.473688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459247+0000","last_change":"2023-10-16T08:18:50.268674+0000","last_active":"2023-10-16T08:18:52.459247+0000","last_peered":"2023-10-16T08:18:52.459247+0000","last_clean":"2023-10-16T08:18:52.459247+0000","last_became_active":"2023-10-16T08:18:50.268413+0000","last_became_peered":"2023-10-16T08:18:50.268413+0000","last_unstale":"2023-10-16T08:18:52.459247+0000","last_undegraded":"2023-10-16T08:18:52.459247+0000","last_fullsized":"2023-10-16T08:18:52.459247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:01.834834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.257440+0000","last_change":"2023-10-16T08:18:49.263362+0000","last_active":"2023-10-16T08:18:50.257440+0000","last_peered":"2023-10-16T08:18:50.257440+0000","last_clean":"2023-10-16T08:18:50.257440+0000","last_became_active":"2023-10-16T08:18:49.262710+0000","last_became_peered":"2023-10-16T08:18:49.262710+0000","last_unstale":"2023-10-16T08:18:50.257440+0000","last_undegraded":"2023-10-16T08:18:50.257440+0000","last_fullsized":"2023-10-16T08:18:50.257440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:51.038960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458852+0000","last_change":"2023-10-16T08:18:50.279270+0000","last_active":"2023-10-16T08:18:52.458852+0000","last_peered":"2023-10-16T08:18:52.458852+0000","last_clean":"2023-10-16T08:18:52.458852+0000","last_became_active":"2023-10-16T08:18:50.279029+0000","last_became_peered":"2023-10-16T08:18:50.279029+0000","last_unstale":"2023-10-16T08:18:52.458852+0000","last_undegraded":"2023-10-16T08:18:52.458852+0000","last_fullsized":"2023-10-16T08:18:52.458852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:13.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.263877+0000","last_change":"2023-10-16T08:18:49.279019+0000","last_active":"2023-10-16T08:18:50.263877+0000","last_peered":"2023-10-16T08:18:50.263877+0000","last_clean":"2023-10-16T08:18:50.263877+0000","last_became_active":"2023-10-16T08:18:49.278310+0000","last_became_peered":"2023-10-16T08:18:49.278310+0000","last_unstale":"2023-10-16T08:18:50.263877+0000","last_undegraded":"2023-10-16T08:18:50.263877+0000","last_fullsized":"2023-10-16T08:18:50.263877+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:59.173567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261598+0000","last_change":"2023-10-16T08:18:50.279006+0000","last_active":"2023-10-16T08:18:51.261598+0000","last_peered":"2023-10-16T08:18:51.261598+0000","last_clean":"2023-10-16T08:18:51.261598+0000","last_became_active":"2023-10-16T08:18:50.278720+0000","last_became_peered":"2023-10-16T08:18:50.278720+0000","last_unstale":"2023-10-16T08:18:51.261598+0000","last_undegraded":"2023-10-16T08:18:51.261598+0000","last_fullsized":"2023-10-16T08:18:51.261598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:13.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.302476+0000","last_change":"2023-10-16T08:18:49.272412+0000","last_active":"2023-10-16T08:18:50.302476+0000","last_peered":"2023-10-16T08:18:50.302476+0000","last_clean":"2023-10-16T08:18:50.302476+0000","last_became_active":"2023-10-16T08:18:49.272131+0000","last_became_peered":"2023-10-16T08:18:49.272131+0000","last_unstale":"2023-10-16T08:18:50.302476+0000","last_undegraded":"2023-10-16T08:18:50.302476+0000","last_fullsized":"2023-10-16T08:18:50.302476+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:01.519805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.262029+0000","last_change":"2023-10-16T08:18:50.277913+0000","last_active":"2023-10-16T08:18:51.262029+0000","last_peered":"2023-10-16T08:18:51.262029+0000","last_clean":"2023-10-16T08:18:51.262029+0000","last_became_active":"2023-10-16T08:18:50.277628+0000","last_became_peered":"2023-10-16T08:18:50.277628+0000","last_unstale":"2023-10-16T08:18:51.262029+0000","last_undegraded":"2023-10-16T08:18:51.262029+0000","last_fullsized":"2023-10-16T08:18:51.262029+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:01.834834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.261886+0000","last_change":"2023-10-16T08:18:49.273714+0000","last_active":"2023-10-16T08:18:50.261886+0000","last_peered":"2023-10-16T08:18:50.261886+0000","last_clean":"2023-10-16T08:18:50.261886+0000","last_became_active":"2023-10-16T08:18:49.273434+0000","last_became_peered":"2023-10-16T08:18:49.273434+0000","last_unstale":"2023-10-16T08:18:50.261886+0000","last_undegraded":"2023-10-16T08:18:50.261886+0000","last_fullsized":"2023-10-16T08:18:50.261886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:24.205203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259732+0000","last_change":"2023-10-16T08:18:50.271084+0000","last_active":"2023-10-16T08:18:51.259732+0000","last_peered":"2023-10-16T08:18:51.259732+0000","last_clean":"2023-10-16T08:18:51.259732+0000","last_became_active":"2023-10-16T08:18:50.270565+0000","last_became_peered":"2023-10-16T08:18:50.270565+0000","last_unstale":"2023-10-16T08:18:51.259732+0000","last_undegraded":"2023-10-16T08:18:51.259732+0000","last_fullsized":"2023-10-16T08:18:51.259732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:58.648055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.260894+0000","last_change":"2023-10-16T08:18:49.275154+0000","last_active":"2023-10-16T08:18:50.260894+0000","last_peered":"2023-10-16T08:18:50.260894+0000","last_clean":"2023-10-16T08:18:50.260894+0000","last_became_active":"2023-10-16T08:18:49.274869+0000","last_became_peered":"2023-10-16T08:18:49.274869+0000","last_unstale":"2023-10-16T08:18:50.260894+0000","last_undegraded":"2023-10-16T08:18:50.260894+0000","last_fullsized":"2023-10-16T08:18:50.260894+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:12:04.702296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459320+0000","last_change":"2023-10-16T08:18:50.272225+0000","last_active":"2023-10-16T08:18:52.459320+0000","last_peered":"2023-10-16T08:18:52.459320+0000","last_clean":"2023-10-16T08:18:52.459320+0000","last_became_active":"2023-10-16T08:18:50.270658+0000","last_became_peered":"2023-10-16T08:18:50.270658+0000","last_unstale":"2023-10-16T08:18:52.459320+0000","last_undegraded":"2023-10-16T08:18:52.459320+0000","last_fullsized":"2023-10-16T08:18:52.459320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:10.454139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.262661+0000","last_change":"2023-10-16T08:18:49.277635+0000","last_active":"2023-10-16T08:18:50.262661+0000","last_peered":"2023-10-16T08:18:50.262661+0000","last_clean":"2023-10-16T08:18:50.262661+0000","last_became_active":"2023-10-16T08:18:49.277326+0000","last_became_peered":"2023-10-16T08:18:49.277326+0000","last_unstale":"2023-10-16T08:18:50.262661+0000","last_undegraded":"2023-10-16T08:18:50.262661+0000","last_fullsized":"2023-10-16T08:18:50.262661+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.847211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114893+0000","last_change":"2023-10-16T08:18:50.274799+0000","last_active":"2023-10-16T08:18:52.114893+0000","last_peered":"2023-10-16T08:18:52.114893+0000","last_clean":"2023-10-16T08:18:52.114893+0000","last_became_active":"2023-10-16T08:18:50.274546+0000","last_became_peered":"2023-10-16T08:18:50.274546+0000","last_unstale":"2023-10-16T08:18:52.114893+0000","last_undegraded":"2023-10-16T08:18:52.114893+0000","last_fullsized":"2023-10-16T08:18:52.114893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:50.360288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.257233+0000","last_change":"2023-10-16T08:18:49.279346+0000","last_active":"2023-10-16T08:18:50.257233+0000","last_peered":"2023-10-16T08:18:50.257233+0000","last_clean":"2023-10-16T08:18:50.257233+0000","last_became_active":"2023-10-16T08:18:49.279132+0000","last_became_peered":"2023-10-16T08:18:49.279132+0000","last_unstale":"2023-10-16T08:18:50.257233+0000","last_undegraded":"2023-10-16T08:18:50.257233+0000","last_fullsized":"2023-10-16T08:18:50.257233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:24.095898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459541+0000","last_change":"2023-10-16T08:18:50.298066+0000","last_active":"2023-10-16T08:18:52.459541+0000","last_peered":"2023-10-16T08:18:52.459541+0000","last_clean":"2023-10-16T08:18:52.459541+0000","last_became_active":"2023-10-16T08:18:50.297839+0000","last_became_peered":"2023-10-16T08:18:50.297839+0000","last_unstale":"2023-10-16T08:18:52.459541+0000","last_undegraded":"2023-10-16T08:18:52.459541+0000","last_fullsized":"2023-10-16T08:18:52.459541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:08.609731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.267381+0000","last_change":"2023-10-16T08:18:49.274856+0000","last_active":"2023-10-16T08:18:50.267381+0000","last_peered":"2023-10-16T08:18:50.267381+0000","last_clean":"2023-10-16T08:18:50.267381+0000","last_became_active":"2023-10-16T08:18:49.274569+0000","last_became_peered":"2023-10-16T08:18:49.274569+0000","last_unstale":"2023-10-16T08:18:50.267381+0000","last_undegraded":"2023-10-16T08:18:50.267381+0000","last_fullsized":"2023-10-16T08:18:50.267381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:12:04.702296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259333+0000","last_change":"2023-10-16T08:18:50.267604+0000","last_active":"2023-10-16T08:18:51.259333+0000","last_peered":"2023-10-16T08:18:51.259333+0000","last_clean":"2023-10-16T08:18:51.259333+0000","last_became_active":"2023-10-16T08:18:50.267308+0000","last_became_peered":"2023-10-16T08:18:50.267308+0000","last_unstale":"2023-10-16T08:18:51.259333+0000","last_undegraded":"2023-10-16T08:18:51.259333+0000","last_fullsized":"2023-10-16T08:18:51.259333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:48.800013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.257645+0000","last_change":"2023-10-16T08:18:49.270112+0000","last_active":"2023-10-16T08:18:50.257645+0000","last_peered":"2023-10-16T08:18:50.257645+0000","last_clean":"2023-10-16T08:18:50.257645+0000","last_became_active":"2023-10-16T08:18:49.269837+0000","last_became_peered":"2023-10-16T08:18:49.269837+0000","last_unstale":"2023-10-16T08:18:50.257645+0000","last_undegraded":"2023-10-16T08:18:50.257645+0000","last_fullsized":"2023-10-16T08:18:50.257645+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:08.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114635+0000","last_change":"2023-10-16T08:18:50.269507+0000","last_active":"2023-10-16T08:18:52.114635+0000","last_peered":"2023-10-16T08:18:52.114635+0000","last_clean":"2023-10-16T08:18:52.114635+0000","last_became_active":"2023-10-16T08:18:50.269221+0000","last_became_peered":"2023-10-16T08:18:50.269221+0000","last_unstale":"2023-10-16T08:18:52.114635+0000","last_undegraded":"2023-10-16T08:18:52.114635+0000","last_fullsized":"2023-10-16T08:18:52.114635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:43.295949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.266016+0000","last_change":"2023-10-16T08:18:49.282764+0000","last_active":"2023-10-16T08:18:50.266016+0000","last_peered":"2023-10-16T08:18:50.266016+0000","last_clean":"2023-10-16T08:18:50.266016+0000","last_became_active":"2023-10-16T08:18:49.282469+0000","last_became_peered":"2023-10-16T08:18:49.282469+0000","last_unstale":"2023-10-16T08:18:50.266016+0000","last_undegraded":"2023-10-16T08:18:50.266016+0000","last_fullsized":"2023-10-16T08:18:50.266016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:56:06.397337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114516+0000","last_change":"2023-10-16T08:18:50.272910+0000","last_active":"2023-10-16T08:18:52.114516+0000","last_peered":"2023-10-16T08:18:52.114516+0000","last_clean":"2023-10-16T08:18:52.114516+0000","last_became_active":"2023-10-16T08:18:50.269982+0000","last_became_peered":"2023-10-16T08:18:50.269982+0000","last_unstale":"2023-10-16T08:18:52.114516+0000","last_undegraded":"2023-10-16T08:18:52.114516+0000","last_fullsized":"2023-10-16T08:18:52.114516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:09.334866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.259321+0000","last_change":"2023-10-16T08:18:49.278621+0000","last_active":"2023-10-16T08:18:50.259321+0000","last_peered":"2023-10-16T08:18:50.259321+0000","last_clean":"2023-10-16T08:18:50.259321+0000","last_became_active":"2023-10-16T08:18:49.278192+0000","last_became_peered":"2023-10-16T08:18:49.278192+0000","last_unstale":"2023-10-16T08:18:50.259321+0000","last_undegraded":"2023-10-16T08:18:50.259321+0000","last_fullsized":"2023-10-16T08:18:50.259321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:11.529725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.269645+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:18:50.269645+0000","last_peered":"2023-10-16T08:18:50.269645+0000","last_clean":"2023-10-16T08:18:50.269645+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:18:50.269645+0000","last_undegraded":"2023-10-16T08:18:50.269645+0000","last_fullsized":"2023-10-16T08:18:50.269645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1249280,"data_stored":879477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82199999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.874,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86699999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93200000000000005,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.53800000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91600000000000004,"5min":1.03,"15min":1.03},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97899999999999998,"5min":1.056,"15min":1.056},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95099999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.57899999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.073,"15min":1.073},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":288949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82899999999999996,"5min":1.077,"15min":1.077},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95399999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72599999999999998},"min":{"1min":0.54200000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.022,"5min":1.343,"15min":1.343},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95499999999999996,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.57899999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.57699999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90400000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.59799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97999999999999998,"5min":1.036,"15min":1.036},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":289601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78100000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.77700000000000002,"5min":2.774,"15min":2.774},"last":0.76700000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84499999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77100000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92500000000000004,"5min":1.099,"15min":1.099},"last":0.88900000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.79600000000000004,"15min":0.82599999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79500000000000004,"5min":2.706,"15min":2.706},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.86699999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.85799999999999998,"15min":0.88},"min":{"1min":0.56599999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97199999999999998,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.875,"15min":0.89900000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95899999999999996,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.93700000000000006}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.495},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.55100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55900000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95099999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64300000000000002}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98499999999999999,"5min":1.002,"15min":1.002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.66900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":290561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75900000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.879}]},{"osd":1,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.82599999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72599999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85999999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.54500000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91100000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.46899999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84199999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.52300000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.86099999999999999,"5min":1.056,"15min":1.056},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":289005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.46300000000000002,"15min":0.47499999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.437},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.629}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82599999999999996,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.51800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84899999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.753},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96099999999999997,"5min":1.355,"15min":1.355},"last":0.80000000000000004}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91900000000000004,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.56100000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92400000000000004,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.85599999999999998,"5min":1.198,"15min":1.198},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.052,"5min":1.325,"15min":1.325},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.82499999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.91400000000000003,"5min":1.391,"15min":1.391},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91100000000000003,"5min":1.3,"15min":1.3},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":289005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.72299999999999998,"15min":0.77000000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0449999999999999,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.433},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.73299999999999998,"15min":0.78300000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.81599999999999995,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.76500000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.73999999999999999,"15min":0.78700000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":4.399,"15min":4.399},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.74199999999999999,"15min":0.78400000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98999999999999999,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.79700000000000004,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.383,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.876,"15min":0.92800000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93000000000000005,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.73499999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.0049999999999999,"15min":1.0309999999999999},"min":{"1min":0.501,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5459999999999998,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.006,"15min":1.0309999999999999},"min":{"1min":0.61699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":3.226,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.94799999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0489999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.0249999999999999,"15min":1.0680000000000001},"min":{"1min":0.57199999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.0369999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.93400000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94199999999999995,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.038,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0069999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.90600000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.038,"15min":1.081},"min":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0600000000000001,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.02,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.878}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":289039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.44500000000000001,"15min":0.45500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64500000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53800000000000003,"5min":1.04,"15min":1.04},"last":0.34300000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.52300000000000002,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.58599999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58899999999999997,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.873,"5min":1.073,"15min":1.073},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59399999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39400000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.82799999999999996,"5min":1.556,"15min":1.556},"last":0.876},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.83099999999999996,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.92300000000000004,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.86199999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.88500000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94399999999999995,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.81999999999999995,"15min":0.83899999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.90100000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.873,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.81200000000000006}]},{"osd":7,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89000000000000001,"15min":0.91300000000000003},"min":{"1min":0.61399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93899999999999995,"5min":1.913,"15min":1.913},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.89700000000000002,"15min":0.92300000000000004},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92700000000000005,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":881511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.38700000000000001,"15min":0.38},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.44800000000000001,"15min":0.42199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.2999999999999998,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58699999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.56399999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.45200000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.55100000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.78400000000000003,"15min":0.746},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.69399999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:18:54.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:18:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.572+0000 7efc7d409640 1 Processor -- start 2023-10-16T08:18:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.572+0000 7efc7d409640 1 -- start start 2023-10-16T08:18:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7efc7815cf00 0x7efc780cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc780d1410 0x7efc780b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc780ce340 con 0x7efc780cdee0 2023-10-16T08:18:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7815d2e0 con 0x7efc780d1410 2023-10-16T08:18:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc780d1930 con 0x7efc7815cf00 2023-10-16T08:18:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc780d1410 0x7efc780b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc6ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efc780cdee0 0x7efc780d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42094/0 (socket says 172.21.15.32:42094) 2023-10-16T08:18:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc6ffff640 1 -- 172.21.15.32:0/2939904091 learned_addr learned my addr 172.21.15.32:0/2939904091 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc76ffd640 1 --2- 172.21.15.32:0/2939904091 >> v2:172.21.15.186:3300/0 conn(0x7efc7815cf00 0x7efc780cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 -- 172.21.15.32:0/2939904091 >> v2:172.21.15.186:3300/0 conn(0x7efc7815cf00 msgr2=0x7efc780cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 --2- 172.21.15.32:0/2939904091 >> v2:172.21.15.186:3300/0 conn(0x7efc7815cf00 0x7efc780cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 -- 172.21.15.32:0/2939904091 >> v1:172.21.15.32:6789/0 conn(0x7efc780cdee0 legacy=0x7efc780d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 -- 172.21.15.32:0/2939904091 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc780d1c10 con 0x7efc780d1410 2023-10-16T08:18:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 --2- 172.21.15.32:0/2939904091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780b6650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efc7806bff0 tx=0x7efc7001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8fa546831caf14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc767fc640 1 -- 172.21.15.32:0/2939904091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc700a5940 con 0x7efc780d1410 2023-10-16T08:18:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc767fc640 1 -- 172.21.15.32:0/2939904091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc700a5aa0 con 0x7efc780d1410 2023-10-16T08:18:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc767fc640 1 -- 172.21.15.32:0/2939904091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc700a5e30 con 0x7efc780d1410 2023-10-16T08:18:54.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- 172.21.15.32:0/2939904091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 msgr2=0x7efc780b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- 172.21.15.32:0/2939904091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780b6650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efc7806bff0 tx=0x7efc7001c880 comp rx=0 tx=0).stop 2023-10-16T08:18:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- 172.21.15.32:0/2939904091 shutdown_connections 2023-10-16T08:18:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- 172.21.15.32:0/2939904091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780b6650 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- 172.21.15.32:0/2939904091 >> v2:172.21.15.186:3300/0 conn(0x7efc7815cf00 0x7efc780cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- 172.21.15.32:0/2939904091 >> 172.21.15.32:0/2939904091 conn(0x7efc780b4470 msgr2=0x7efc780b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- 172.21.15.32:0/2939904091 shutdown_connections 2023-10-16T08:18:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- 172.21.15.32:0/2939904091 wait complete. 2023-10-16T08:18:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 Processor -- start 2023-10-16T08:18:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- start start 2023-10-16T08:18:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc780cdee0 0x7efc780ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7815cf00 0x7efc78155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780ab010 con 0x7efc780d1410 2023-10-16T08:18:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780ab500 con 0x7efc7815cf00 2023-10-16T08:18:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc7d409640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc780ab7e0 con 0x7efc780cdee0 2023-10-16T08:18:54.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780aaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43570/0 (socket says 172.21.15.32:43570) 2023-10-16T08:18:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc6ffff640 1 -- 172.21.15.32:0/2621009877 learned_addr learned my addr 172.21.15.32:0/2621009877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc76ffd640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc780cdee0 0x7efc780ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.576+0000 7efc777fe640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7815cf00 0x7efc78155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6ffff640 1 -- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc780cdee0 msgr2=0x7efc780ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6ffff640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc780cdee0 0x7efc780ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6ffff640 1 -- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7815cf00 msgr2=0x7efc78155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6ffff640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7815cf00 0x7efc78155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6ffff640 1 -- 172.21.15.32:0/2621009877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70009a70 con 0x7efc780d1410 2023-10-16T08:18:54.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc777fe640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7815cf00 0x7efc78155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6ffff640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780aaad0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c500 tx=0x7efc6000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60005cf0 con 0x7efc780d1410 2023-10-16T08:18:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc60005e50 con 0x7efc780d1410 2023-10-16T08:18:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc600072f0 con 0x7efc780d1410 2023-10-16T08:18:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc780abac0 con 0x7efc780d1410 2023-10-16T08:18:54.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781564f0 con 0x7efc780d1410 2023-10-16T08:18:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc780abfe0 con 0x7efc780d1410 2023-10-16T08:18:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc60007450 con 0x7efc780d1410 2023-10-16T08:18:54.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6f7fe640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc580664a0 0x7efc58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7efc60083490 con 0x7efc780d1410 2023-10-16T08:18:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.580+0000 7efc76ffd640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc580664a0 0x7efc58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.584+0000 7efc76ffd640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc580664a0 0x7efc58068960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc781f8a70 tx=0x7efc68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.584+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc780abfe0 con 0x7efc780d1410 2023-10-16T08:18:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.740+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc780b5de0 con 0x7efc580664a0 2023-10-16T08:18:54.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc6f7fe640 1 -- 172.21.15.32:0/2621009877 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289477 (secure 0 0 0) 0x7efc680118e0 con 0x7efc580664a0 2023-10-16T08:18:54.758 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:54.760 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:18:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc580664a0 msgr2=0x7efc58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc580664a0 0x7efc58068960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc781f8a70 tx=0x7efc68008040 comp rx=0 tx=0).stop 2023-10-16T08:18:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 msgr2=0x7efc780aaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780aaad0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c500 tx=0x7efc6000c9c0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 shutdown_connections 2023-10-16T08:18:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7815cf00 0x7efc78155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc780d1410 0x7efc780aaad0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7efc580664a0 0x7efc58068960 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 --2- 172.21.15.32:0/2621009877 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7efc780cdee0 0x7efc780ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 >> 172.21.15.32:0/2621009877 conn(0x7efc780b4470 msgr2=0x7efc780b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 shutdown_connections 2023-10-16T08:18:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.752+0000 7efc7d409640 1 -- 172.21.15.32:0/2621009877 wait complete. 2023-10-16T08:18:54.779 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-10-16T08:18:54.050670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239108,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735612,"statfs":{"total":773094113280,"available":772849266688,"internally_reserved":0,"allocated":6430720,"data_stored":3497148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.806592"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036516+0000","last_change":"2023-10-16T08:18:50.293165+0000","last_active":"2023-10-16T08:18:52.036516+0000","last_peered":"2023-10-16T08:18:52.036516+0000","last_clean":"2023-10-16T08:18:52.036516+0000","last_became_active":"2023-10-16T08:18:50.292843+0000","last_became_peered":"2023-10-16T08:18:50.292843+0000","last_unstale":"2023-10-16T08:18:52.036516+0000","last_undegraded":"2023-10-16T08:18:52.036516+0000","last_fullsized":"2023-10-16T08:18:52.036516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:53.897776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114910+0000","last_change":"2023-10-16T08:18:50.290413+0000","last_active":"2023-10-16T08:18:52.114910+0000","last_peered":"2023-10-16T08:18:52.114910+0000","last_clean":"2023-10-16T08:18:52.114910+0000","last_became_active":"2023-10-16T08:18:50.290261+0000","last_became_peered":"2023-10-16T08:18:50.290261+0000","last_unstale":"2023-10-16T08:18:52.114910+0000","last_undegraded":"2023-10-16T08:18:52.114910+0000","last_fullsized":"2023-10-16T08:18:52.114910+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:43:35.553681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459572+0000","last_change":"2023-10-16T08:18:50.284610+0000","last_active":"2023-10-16T08:18:52.459572+0000","last_peered":"2023-10-16T08:18:52.459572+0000","last_clean":"2023-10-16T08:18:52.459572+0000","last_became_active":"2023-10-16T08:18:50.284331+0000","last_became_peered":"2023-10-16T08:18:50.284331+0000","last_unstale":"2023-10-16T08:18:52.459572+0000","last_undegraded":"2023-10-16T08:18:52.459572+0000","last_fullsized":"2023-10-16T08:18:52.459572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:38.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261764+0000","last_change":"2023-10-16T08:18:50.278407+0000","last_active":"2023-10-16T08:18:51.261764+0000","last_peered":"2023-10-16T08:18:51.261764+0000","last_clean":"2023-10-16T08:18:51.261764+0000","last_became_active":"2023-10-16T08:18:50.278143+0000","last_became_peered":"2023-10-16T08:18:50.278143+0000","last_unstale":"2023-10-16T08:18:51.261764+0000","last_undegraded":"2023-10-16T08:18:51.261764+0000","last_fullsized":"2023-10-16T08:18:51.261764+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:08.609731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036332+0000","last_change":"2023-10-16T08:18:50.279422+0000","last_active":"2023-10-16T08:18:52.036332+0000","last_peered":"2023-10-16T08:18:52.036332+0000","last_clean":"2023-10-16T08:18:52.036332+0000","last_became_active":"2023-10-16T08:18:50.279185+0000","last_became_peered":"2023-10-16T08:18:50.279185+0000","last_unstale":"2023-10-16T08:18:52.036332+0000","last_undegraded":"2023-10-16T08:18:52.036332+0000","last_fullsized":"2023-10-16T08:18:52.036332+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:32.567469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458867+0000","last_change":"2023-10-16T08:18:50.272152+0000","last_active":"2023-10-16T08:18:52.458867+0000","last_peered":"2023-10-16T08:18:52.458867+0000","last_clean":"2023-10-16T08:18:52.458867+0000","last_became_active":"2023-10-16T08:18:50.270371+0000","last_became_peered":"2023-10-16T08:18:50.270371+0000","last_unstale":"2023-10-16T08:18:52.458867+0000","last_undegraded":"2023-10-16T08:18:52.458867+0000","last_fullsized":"2023-10-16T08:18:52.458867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:38.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653438+0000","last_change":"2023-10-16T08:18:50.284250+0000","last_active":"2023-10-16T08:18:51.653438+0000","last_peered":"2023-10-16T08:18:51.653438+0000","last_clean":"2023-10-16T08:18:51.653438+0000","last_became_active":"2023-10-16T08:18:50.283263+0000","last_became_peered":"2023-10-16T08:18:50.283263+0000","last_unstale":"2023-10-16T08:18:51.653438+0000","last_undegraded":"2023-10-16T08:18:51.653438+0000","last_fullsized":"2023-10-16T08:18:51.653438+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:46:40.534100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114412+0000","last_change":"2023-10-16T08:18:50.275182+0000","last_active":"2023-10-16T08:18:52.114412+0000","last_peered":"2023-10-16T08:18:52.114412+0000","last_clean":"2023-10-16T08:18:52.114412+0000","last_became_active":"2023-10-16T08:18:50.274932+0000","last_became_peered":"2023-10-16T08:18:50.274932+0000","last_unstale":"2023-10-16T08:18:52.114412+0000","last_undegraded":"2023-10-16T08:18:52.114412+0000","last_fullsized":"2023-10-16T08:18:52.114412+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:29:30.268242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653858+0000","last_change":"2023-10-16T08:18:50.285316+0000","last_active":"2023-10-16T08:18:51.653858+0000","last_peered":"2023-10-16T08:18:51.653858+0000","last_clean":"2023-10-16T08:18:51.653858+0000","last_became_active":"2023-10-16T08:18:50.285108+0000","last_became_peered":"2023-10-16T08:18:50.285108+0000","last_unstale":"2023-10-16T08:18:51.653858+0000","last_undegraded":"2023-10-16T08:18:51.653858+0000","last_fullsized":"2023-10-16T08:18:51.653858+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:45.628065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259995+0000","last_change":"2023-10-16T08:18:50.277091+0000","last_active":"2023-10-16T08:18:51.259995+0000","last_peered":"2023-10-16T08:18:51.259995+0000","last_clean":"2023-10-16T08:18:51.259995+0000","last_became_active":"2023-10-16T08:18:50.276777+0000","last_became_peered":"2023-10-16T08:18:50.276777+0000","last_unstale":"2023-10-16T08:18:51.259995+0000","last_undegraded":"2023-10-16T08:18:51.259995+0000","last_fullsized":"2023-10-16T08:18:51.259995+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:17.724968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259966+0000","last_change":"2023-10-16T08:18:50.290128+0000","last_active":"2023-10-16T08:18:51.259966+0000","last_peered":"2023-10-16T08:18:51.259966+0000","last_clean":"2023-10-16T08:18:51.259966+0000","last_became_active":"2023-10-16T08:18:50.289788+0000","last_became_peered":"2023-10-16T08:18:50.289788+0000","last_unstale":"2023-10-16T08:18:51.259966+0000","last_undegraded":"2023-10-16T08:18:51.259966+0000","last_fullsized":"2023-10-16T08:18:51.259966+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:28.307736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261846+0000","last_change":"2023-10-16T08:18:50.278051+0000","last_active":"2023-10-16T08:18:51.261846+0000","last_peered":"2023-10-16T08:18:51.261846+0000","last_clean":"2023-10-16T08:18:51.261846+0000","last_became_active":"2023-10-16T08:18:50.277791+0000","last_became_peered":"2023-10-16T08:18:50.277791+0000","last_unstale":"2023-10-16T08:18:51.261846+0000","last_undegraded":"2023-10-16T08:18:51.261846+0000","last_fullsized":"2023-10-16T08:18:51.261846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:38.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.262011+0000","last_change":"2023-10-16T08:18:50.274455+0000","last_active":"2023-10-16T08:18:51.262011+0000","last_peered":"2023-10-16T08:18:51.262011+0000","last_clean":"2023-10-16T08:18:51.262011+0000","last_became_active":"2023-10-16T08:18:50.274172+0000","last_became_peered":"2023-10-16T08:18:50.274172+0000","last_unstale":"2023-10-16T08:18:51.262011+0000","last_undegraded":"2023-10-16T08:18:51.262011+0000","last_fullsized":"2023-10-16T08:18:51.262011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:53.028262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653632+0000","last_change":"2023-10-16T08:18:50.283561+0000","last_active":"2023-10-16T08:18:51.653632+0000","last_peered":"2023-10-16T08:18:51.653632+0000","last_clean":"2023-10-16T08:18:51.653632+0000","last_became_active":"2023-10-16T08:18:50.283289+0000","last_became_peered":"2023-10-16T08:18:50.283289+0000","last_unstale":"2023-10-16T08:18:51.653632+0000","last_undegraded":"2023-10-16T08:18:51.653632+0000","last_fullsized":"2023-10-16T08:18:51.653632+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:58.648055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653560+0000","last_change":"2023-10-16T08:18:50.275300+0000","last_active":"2023-10-16T08:18:51.653560+0000","last_peered":"2023-10-16T08:18:51.653560+0000","last_clean":"2023-10-16T08:18:51.653560+0000","last_became_active":"2023-10-16T08:18:50.274924+0000","last_became_peered":"2023-10-16T08:18:50.274924+0000","last_unstale":"2023-10-16T08:18:51.653560+0000","last_undegraded":"2023-10-16T08:18:51.653560+0000","last_fullsized":"2023-10-16T08:18:51.653560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:48.800013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458798+0000","last_change":"2023-10-16T08:18:50.268536+0000","last_active":"2023-10-16T08:18:52.458798+0000","last_peered":"2023-10-16T08:18:52.458798+0000","last_clean":"2023-10-16T08:18:52.458798+0000","last_became_active":"2023-10-16T08:18:50.268265+0000","last_became_peered":"2023-10-16T08:18:50.268265+0000","last_unstale":"2023-10-16T08:18:52.458798+0000","last_undegraded":"2023-10-16T08:18:52.458798+0000","last_fullsized":"2023-10-16T08:18:52.458798+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:13.623970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.115127+0000","last_change":"2023-10-16T08:18:50.276117+0000","last_active":"2023-10-16T08:18:52.115127+0000","last_peered":"2023-10-16T08:18:52.115127+0000","last_clean":"2023-10-16T08:18:52.115127+0000","last_became_active":"2023-10-16T08:18:50.275943+0000","last_became_peered":"2023-10-16T08:18:50.275943+0000","last_unstale":"2023-10-16T08:18:52.115127+0000","last_undegraded":"2023-10-16T08:18:52.115127+0000","last_fullsized":"2023-10-16T08:18:52.115127+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:51:18.861708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459714+0000","last_change":"2023-10-16T08:18:50.285032+0000","last_active":"2023-10-16T08:18:52.459714+0000","last_peered":"2023-10-16T08:18:52.459714+0000","last_clean":"2023-10-16T08:18:52.459714+0000","last_became_active":"2023-10-16T08:18:50.284742+0000","last_became_peered":"2023-10-16T08:18:50.284742+0000","last_unstale":"2023-10-16T08:18:52.459714+0000","last_undegraded":"2023-10-16T08:18:52.459714+0000","last_fullsized":"2023-10-16T08:18:52.459714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:36.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036325+0000","last_change":"2023-10-16T08:18:50.303134+0000","last_active":"2023-10-16T08:18:52.036325+0000","last_peered":"2023-10-16T08:18:52.036325+0000","last_clean":"2023-10-16T08:18:52.036325+0000","last_became_active":"2023-10-16T08:18:50.302843+0000","last_became_peered":"2023-10-16T08:18:50.302843+0000","last_unstale":"2023-10-16T08:18:52.036325+0000","last_undegraded":"2023-10-16T08:18:52.036325+0000","last_fullsized":"2023-10-16T08:18:52.036325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:33.173663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114852+0000","last_change":"2023-10-16T08:18:50.280459+0000","last_active":"2023-10-16T08:18:52.114852+0000","last_peered":"2023-10-16T08:18:52.114852+0000","last_clean":"2023-10-16T08:18:52.114852+0000","last_became_active":"2023-10-16T08:18:50.280283+0000","last_became_peered":"2023-10-16T08:18:50.280283+0000","last_unstale":"2023-10-16T08:18:52.114852+0000","last_undegraded":"2023-10-16T08:18:52.114852+0000","last_fullsized":"2023-10-16T08:18:52.114852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:58.958197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459262+0000","last_change":"2023-10-16T08:18:50.285379+0000","last_active":"2023-10-16T08:18:52.459262+0000","last_peered":"2023-10-16T08:18:52.459262+0000","last_clean":"2023-10-16T08:18:52.459262+0000","last_became_active":"2023-10-16T08:18:50.285090+0000","last_became_peered":"2023-10-16T08:18:50.285090+0000","last_unstale":"2023-10-16T08:18:52.459262+0000","last_undegraded":"2023-10-16T08:18:52.459262+0000","last_fullsized":"2023-10-16T08:18:52.459262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:09.144644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114483+0000","last_change":"2023-10-16T08:18:50.291868+0000","last_active":"2023-10-16T08:18:52.114483+0000","last_peered":"2023-10-16T08:18:52.114483+0000","last_clean":"2023-10-16T08:18:52.114483+0000","last_became_active":"2023-10-16T08:18:50.286778+0000","last_became_peered":"2023-10-16T08:18:50.286778+0000","last_unstale":"2023-10-16T08:18:52.114483+0000","last_undegraded":"2023-10-16T08:18:52.114483+0000","last_fullsized":"2023-10-16T08:18:52.114483+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:11:53.180528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459229+0000","last_change":"2023-10-16T08:18:50.287810+0000","last_active":"2023-10-16T08:18:52.459229+0000","last_peered":"2023-10-16T08:18:52.459229+0000","last_clean":"2023-10-16T08:18:52.459229+0000","last_became_active":"2023-10-16T08:18:50.287378+0000","last_became_peered":"2023-10-16T08:18:50.287378+0000","last_unstale":"2023-10-16T08:18:52.459229+0000","last_undegraded":"2023-10-16T08:18:52.459229+0000","last_fullsized":"2023-10-16T08:18:52.459229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:49.807775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261185+0000","last_change":"2023-10-16T08:18:50.273131+0000","last_active":"2023-10-16T08:18:51.261185+0000","last_peered":"2023-10-16T08:18:51.261185+0000","last_clean":"2023-10-16T08:18:51.261185+0000","last_became_active":"2023-10-16T08:18:50.272572+0000","last_became_peered":"2023-10-16T08:18:50.272572+0000","last_unstale":"2023-10-16T08:18:51.261185+0000","last_undegraded":"2023-10-16T08:18:51.261185+0000","last_fullsized":"2023-10-16T08:18:51.261185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:26:34.317445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036575+0000","last_change":"2023-10-16T08:18:50.299812+0000","last_active":"2023-10-16T08:18:52.036575+0000","last_peered":"2023-10-16T08:18:52.036575+0000","last_clean":"2023-10-16T08:18:52.036575+0000","last_became_active":"2023-10-16T08:18:50.299497+0000","last_became_peered":"2023-10-16T08:18:50.299497+0000","last_unstale":"2023-10-16T08:18:52.036575+0000","last_undegraded":"2023-10-16T08:18:52.036575+0000","last_fullsized":"2023-10-16T08:18:52.036575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:57.670386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261444+0000","last_change":"2023-10-16T08:18:50.271399+0000","last_active":"2023-10-16T08:18:51.261444+0000","last_peered":"2023-10-16T08:18:51.261444+0000","last_clean":"2023-10-16T08:18:51.261444+0000","last_became_active":"2023-10-16T08:18:50.271125+0000","last_became_peered":"2023-10-16T08:18:50.271125+0000","last_unstale":"2023-10-16T08:18:51.261444+0000","last_undegraded":"2023-10-16T08:18:51.261444+0000","last_fullsized":"2023-10-16T08:18:51.261444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:13.623970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459198+0000","last_change":"2023-10-16T08:18:50.272337+0000","last_active":"2023-10-16T08:18:52.459198+0000","last_peered":"2023-10-16T08:18:52.459198+0000","last_clean":"2023-10-16T08:18:52.459198+0000","last_became_active":"2023-10-16T08:18:50.271938+0000","last_became_peered":"2023-10-16T08:18:50.271938+0000","last_unstale":"2023-10-16T08:18:52.459198+0000","last_undegraded":"2023-10-16T08:18:52.459198+0000","last_fullsized":"2023-10-16T08:18:52.459198+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:36.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036384+0000","last_change":"2023-10-16T08:18:50.279472+0000","last_active":"2023-10-16T08:18:52.036384+0000","last_peered":"2023-10-16T08:18:52.036384+0000","last_clean":"2023-10-16T08:18:52.036384+0000","last_became_active":"2023-10-16T08:18:50.279171+0000","last_became_peered":"2023-10-16T08:18:50.279171+0000","last_unstale":"2023-10-16T08:18:52.036384+0000","last_undegraded":"2023-10-16T08:18:52.036384+0000","last_fullsized":"2023-10-16T08:18:52.036384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:13.057328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114847+0000","last_change":"2023-10-16T08:18:50.275373+0000","last_active":"2023-10-16T08:18:52.114847+0000","last_peered":"2023-10-16T08:18:52.114847+0000","last_clean":"2023-10-16T08:18:52.114847+0000","last_became_active":"2023-10-16T08:18:50.275101+0000","last_became_peered":"2023-10-16T08:18:50.275101+0000","last_unstale":"2023-10-16T08:18:52.114847+0000","last_undegraded":"2023-10-16T08:18:52.114847+0000","last_fullsized":"2023-10-16T08:18:52.114847+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:34.987039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459323+0000","last_change":"2023-10-16T08:18:50.282594+0000","last_active":"2023-10-16T08:18:52.459323+0000","last_peered":"2023-10-16T08:18:52.459323+0000","last_clean":"2023-10-16T08:18:52.459323+0000","last_became_active":"2023-10-16T08:18:50.282378+0000","last_became_peered":"2023-10-16T08:18:50.282378+0000","last_unstale":"2023-10-16T08:18:52.459323+0000","last_undegraded":"2023-10-16T08:18:52.459323+0000","last_fullsized":"2023-10-16T08:18:52.459323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:10:00.180477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.036269+0000","last_change":"2023-10-16T08:18:50.297269+0000","last_active":"2023-10-16T08:18:52.036269+0000","last_peered":"2023-10-16T08:18:52.036269+0000","last_clean":"2023-10-16T08:18:52.036269+0000","last_became_active":"2023-10-16T08:18:50.296997+0000","last_became_peered":"2023-10-16T08:18:50.296997+0000","last_unstale":"2023-10-16T08:18:52.036269+0000","last_undegraded":"2023-10-16T08:18:52.036269+0000","last_fullsized":"2023-10-16T08:18:52.036269+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:58.116685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458867+0000","last_change":"2023-10-16T08:18:50.278627+0000","last_active":"2023-10-16T08:18:52.458867+0000","last_peered":"2023-10-16T08:18:52.458867+0000","last_clean":"2023-10-16T08:18:52.458867+0000","last_became_active":"2023-10-16T08:18:50.278340+0000","last_became_peered":"2023-10-16T08:18:50.278340+0000","last_unstale":"2023-10-16T08:18:52.458867+0000","last_undegraded":"2023-10-16T08:18:52.458867+0000","last_fullsized":"2023-10-16T08:18:52.458867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:08.609731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459163+0000","last_change":"2023-10-16T08:18:50.280362+0000","last_active":"2023-10-16T08:18:52.459163+0000","last_peered":"2023-10-16T08:18:52.459163+0000","last_clean":"2023-10-16T08:18:52.459163+0000","last_became_active":"2023-10-16T08:18:50.280096+0000","last_became_peered":"2023-10-16T08:18:50.280096+0000","last_unstale":"2023-10-16T08:18:52.459163+0000","last_undegraded":"2023-10-16T08:18:52.459163+0000","last_fullsized":"2023-10-16T08:18:52.459163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:18.275313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.301561+0000","last_change":"2023-10-16T08:18:49.280394+0000","last_active":"2023-10-16T08:18:50.301561+0000","last_peered":"2023-10-16T08:18:50.301561+0000","last_clean":"2023-10-16T08:18:50.301561+0000","last_became_active":"2023-10-16T08:18:49.280087+0000","last_became_peered":"2023-10-16T08:18:49.280087+0000","last_unstale":"2023-10-16T08:18:50.301561+0000","last_undegraded":"2023-10-16T08:18:50.301561+0000","last_fullsized":"2023-10-16T08:18:50.301561+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:27.300826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.260052+0000","last_change":"2023-10-16T08:18:50.276206+0000","last_active":"2023-10-16T08:18:51.260052+0000","last_peered":"2023-10-16T08:18:51.260052+0000","last_clean":"2023-10-16T08:18:51.260052+0000","last_became_active":"2023-10-16T08:18:50.275914+0000","last_became_peered":"2023-10-16T08:18:50.275914+0000","last_unstale":"2023-10-16T08:18:51.260052+0000","last_undegraded":"2023-10-16T08:18:51.260052+0000","last_fullsized":"2023-10-16T08:18:51.260052+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:19:06.070977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.297199+0000","last_change":"2023-10-16T08:18:49.283430+0000","last_active":"2023-10-16T08:18:50.297199+0000","last_peered":"2023-10-16T08:18:50.297199+0000","last_clean":"2023-10-16T08:18:50.297199+0000","last_became_active":"2023-10-16T08:18:49.283116+0000","last_became_peered":"2023-10-16T08:18:49.283116+0000","last_unstale":"2023-10-16T08:18:50.297199+0000","last_undegraded":"2023-10-16T08:18:50.297199+0000","last_fullsized":"2023-10-16T08:18:50.297199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:15.786640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459338+0000","last_change":"2023-10-16T08:18:50.271957+0000","last_active":"2023-10-16T08:18:52.459338+0000","last_peered":"2023-10-16T08:18:52.459338+0000","last_clean":"2023-10-16T08:18:52.459338+0000","last_became_active":"2023-10-16T08:18:50.269540+0000","last_became_peered":"2023-10-16T08:18:50.269540+0000","last_unstale":"2023-10-16T08:18:52.459338+0000","last_undegraded":"2023-10-16T08:18:52.459338+0000","last_fullsized":"2023-10-16T08:18:52.459338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:56:07.404247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.302520+0000","last_change":"2023-10-16T08:18:49.275196+0000","last_active":"2023-10-16T08:18:50.302520+0000","last_peered":"2023-10-16T08:18:50.302520+0000","last_clean":"2023-10-16T08:18:50.302520+0000","last_became_active":"2023-10-16T08:18:49.274931+0000","last_became_peered":"2023-10-16T08:18:49.274931+0000","last_unstale":"2023-10-16T08:18:50.302520+0000","last_undegraded":"2023-10-16T08:18:50.302520+0000","last_fullsized":"2023-10-16T08:18:50.302520+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:59.173567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459295+0000","last_change":"2023-10-16T08:18:50.282434+0000","last_active":"2023-10-16T08:18:52.459295+0000","last_peered":"2023-10-16T08:18:52.459295+0000","last_clean":"2023-10-16T08:18:52.459295+0000","last_became_active":"2023-10-16T08:18:50.281408+0000","last_became_peered":"2023-10-16T08:18:50.281408+0000","last_unstale":"2023-10-16T08:18:52.459295+0000","last_undegraded":"2023-10-16T08:18:52.459295+0000","last_fullsized":"2023-10-16T08:18:52.459295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:54.136989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296203+0000","last_change":"2023-10-16T08:18:49.279763+0000","last_active":"2023-10-16T08:18:50.296203+0000","last_peered":"2023-10-16T08:18:50.296203+0000","last_clean":"2023-10-16T08:18:50.296203+0000","last_became_active":"2023-10-16T08:18:49.279483+0000","last_became_peered":"2023-10-16T08:18:49.279483+0000","last_unstale":"2023-10-16T08:18:50.296203+0000","last_undegraded":"2023-10-16T08:18:50.296203+0000","last_fullsized":"2023-10-16T08:18:50.296203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:39.925292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114477+0000","last_change":"2023-10-16T08:18:50.286758+0000","last_active":"2023-10-16T08:18:52.114477+0000","last_peered":"2023-10-16T08:18:52.114477+0000","last_clean":"2023-10-16T08:18:52.114477+0000","last_became_active":"2023-10-16T08:18:50.286487+0000","last_became_peered":"2023-10-16T08:18:50.286487+0000","last_unstale":"2023-10-16T08:18:52.114477+0000","last_undegraded":"2023-10-16T08:18:52.114477+0000","last_fullsized":"2023-10-16T08:18:52.114477+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:26.466105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.295708+0000","last_change":"2023-10-16T08:18:49.270849+0000","last_active":"2023-10-16T08:18:50.295708+0000","last_peered":"2023-10-16T08:18:50.295708+0000","last_clean":"2023-10-16T08:18:50.295708+0000","last_became_active":"2023-10-16T08:18:49.270148+0000","last_became_peered":"2023-10-16T08:18:49.270148+0000","last_unstale":"2023-10-16T08:18:50.295708+0000","last_undegraded":"2023-10-16T08:18:50.295708+0000","last_fullsized":"2023-10-16T08:18:50.295708+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:08.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114643+0000","last_change":"2023-10-16T08:18:50.272082+0000","last_active":"2023-10-16T08:18:52.114643+0000","last_peered":"2023-10-16T08:18:52.114643+0000","last_clean":"2023-10-16T08:18:52.114643+0000","last_became_active":"2023-10-16T08:18:50.271809+0000","last_became_peered":"2023-10-16T08:18:50.271809+0000","last_unstale":"2023-10-16T08:18:52.114643+0000","last_undegraded":"2023-10-16T08:18:52.114643+0000","last_fullsized":"2023-10-16T08:18:52.114643+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:22.987216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296324+0000","last_change":"2023-10-16T08:18:49.272914+0000","last_active":"2023-10-16T08:18:50.296324+0000","last_peered":"2023-10-16T08:18:50.296324+0000","last_clean":"2023-10-16T08:18:50.296324+0000","last_became_active":"2023-10-16T08:18:49.272652+0000","last_became_peered":"2023-10-16T08:18:49.272652+0000","last_unstale":"2023-10-16T08:18:50.296324+0000","last_undegraded":"2023-10-16T08:18:50.296324+0000","last_fullsized":"2023-10-16T08:18:50.296324+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:16.718057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261334+0000","last_change":"2023-10-16T08:18:50.275700+0000","last_active":"2023-10-16T08:18:51.261334+0000","last_peered":"2023-10-16T08:18:51.261334+0000","last_clean":"2023-10-16T08:18:51.261334+0000","last_became_active":"2023-10-16T08:18:50.275442+0000","last_became_peered":"2023-10-16T08:18:50.275442+0000","last_unstale":"2023-10-16T08:18:51.261334+0000","last_undegraded":"2023-10-16T08:18:51.261334+0000","last_fullsized":"2023-10-16T08:18:51.261334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:09.092916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.262597+0000","last_change":"2023-10-16T08:18:49.279275+0000","last_active":"2023-10-16T08:18:50.262597+0000","last_peered":"2023-10-16T08:18:50.262597+0000","last_clean":"2023-10-16T08:18:50.262597+0000","last_became_active":"2023-10-16T08:18:49.278699+0000","last_became_peered":"2023-10-16T08:18:49.278699+0000","last_unstale":"2023-10-16T08:18:50.262597+0000","last_undegraded":"2023-10-16T08:18:50.262597+0000","last_fullsized":"2023-10-16T08:18:50.262597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:24.205203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114572+0000","last_change":"2023-10-16T08:18:50.277601+0000","last_active":"2023-10-16T08:18:52.114572+0000","last_peered":"2023-10-16T08:18:52.114572+0000","last_clean":"2023-10-16T08:18:52.114572+0000","last_became_active":"2023-10-16T08:18:50.277357+0000","last_became_peered":"2023-10-16T08:18:50.277357+0000","last_unstale":"2023-10-16T08:18:52.114572+0000","last_undegraded":"2023-10-16T08:18:52.114572+0000","last_fullsized":"2023-10-16T08:18:52.114572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:21.664383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.265797+0000","last_change":"2023-10-16T08:18:49.278010+0000","last_active":"2023-10-16T08:18:50.265797+0000","last_peered":"2023-10-16T08:18:50.265797+0000","last_clean":"2023-10-16T08:18:50.265797+0000","last_became_active":"2023-10-16T08:18:49.277690+0000","last_became_peered":"2023-10-16T08:18:49.277690+0000","last_unstale":"2023-10-16T08:18:50.265797+0000","last_undegraded":"2023-10-16T08:18:50.265797+0000","last_fullsized":"2023-10-16T08:18:50.265797+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:10:22.077040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653865+0000","last_change":"2023-10-16T08:18:50.280582+0000","last_active":"2023-10-16T08:18:51.653865+0000","last_peered":"2023-10-16T08:18:51.653865+0000","last_clean":"2023-10-16T08:18:51.653865+0000","last_became_active":"2023-10-16T08:18:50.280318+0000","last_became_peered":"2023-10-16T08:18:50.280318+0000","last_unstale":"2023-10-16T08:18:51.653865+0000","last_undegraded":"2023-10-16T08:18:51.653865+0000","last_fullsized":"2023-10-16T08:18:51.653865+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:28.307736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.273847+0000","last_change":"2023-10-16T08:18:49.282731+0000","last_active":"2023-10-16T08:18:50.273847+0000","last_peered":"2023-10-16T08:18:50.273847+0000","last_clean":"2023-10-16T08:18:50.273847+0000","last_became_active":"2023-10-16T08:18:49.282408+0000","last_became_peered":"2023-10-16T08:18:49.282408+0000","last_unstale":"2023-10-16T08:18:50.273847+0000","last_undegraded":"2023-10-16T08:18:50.273847+0000","last_fullsized":"2023-10-16T08:18:50.273847+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.847211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459812+0000","last_change":"2023-10-16T08:18:50.287101+0000","last_active":"2023-10-16T08:18:52.459812+0000","last_peered":"2023-10-16T08:18:52.459812+0000","last_clean":"2023-10-16T08:18:52.459812+0000","last_became_active":"2023-10-16T08:18:50.286807+0000","last_became_peered":"2023-10-16T08:18:50.286807+0000","last_unstale":"2023-10-16T08:18:52.459812+0000","last_undegraded":"2023-10-16T08:18:52.459812+0000","last_fullsized":"2023-10-16T08:18:52.459812+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:10.454139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.267966+0000","last_change":"2023-10-16T08:18:49.276851+0000","last_active":"2023-10-16T08:18:50.267966+0000","last_peered":"2023-10-16T08:18:50.267966+0000","last_clean":"2023-10-16T08:18:50.267966+0000","last_became_active":"2023-10-16T08:18:49.276615+0000","last_became_peered":"2023-10-16T08:18:49.276615+0000","last_unstale":"2023-10-16T08:18:50.267966+0000","last_undegraded":"2023-10-16T08:18:50.267966+0000","last_fullsized":"2023-10-16T08:18:50.267966+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:29.702452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259756+0000","last_change":"2023-10-16T08:18:50.271193+0000","last_active":"2023-10-16T08:18:51.259756+0000","last_peered":"2023-10-16T08:18:51.259756+0000","last_clean":"2023-10-16T08:18:51.259756+0000","last_became_active":"2023-10-16T08:18:50.270850+0000","last_became_peered":"2023-10-16T08:18:50.270850+0000","last_unstale":"2023-10-16T08:18:51.259756+0000","last_undegraded":"2023-10-16T08:18:51.259756+0000","last_fullsized":"2023-10-16T08:18:51.259756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:45.628065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.299274+0000","last_change":"2023-10-16T08:18:49.283642+0000","last_active":"2023-10-16T08:18:50.299274+0000","last_peered":"2023-10-16T08:18:50.299274+0000","last_clean":"2023-10-16T08:18:50.299274+0000","last_became_active":"2023-10-16T08:18:49.283364+0000","last_became_peered":"2023-10-16T08:18:49.283364+0000","last_unstale":"2023-10-16T08:18:50.299274+0000","last_undegraded":"2023-10-16T08:18:50.299274+0000","last_fullsized":"2023-10-16T08:18:50.299274+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:39.925292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459598+0000","last_change":"2023-10-16T08:18:50.297192+0000","last_active":"2023-10-16T08:18:52.459598+0000","last_peered":"2023-10-16T08:18:52.459598+0000","last_clean":"2023-10-16T08:18:52.459598+0000","last_became_active":"2023-10-16T08:18:50.296934+0000","last_became_peered":"2023-10-16T08:18:50.296934+0000","last_unstale":"2023-10-16T08:18:52.459598+0000","last_undegraded":"2023-10-16T08:18:52.459598+0000","last_fullsized":"2023-10-16T08:18:52.459598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:18.275313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.302553+0000","last_change":"2023-10-16T08:18:49.276927+0000","last_active":"2023-10-16T08:18:50.302553+0000","last_peered":"2023-10-16T08:18:50.302553+0000","last_clean":"2023-10-16T08:18:50.302553+0000","last_became_active":"2023-10-16T08:18:49.276611+0000","last_became_peered":"2023-10-16T08:18:49.276611+0000","last_unstale":"2023-10-16T08:18:50.302553+0000","last_undegraded":"2023-10-16T08:18:50.302553+0000","last_fullsized":"2023-10-16T08:18:50.302553+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:51.038960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261904+0000","last_change":"2023-10-16T08:18:50.269002+0000","last_active":"2023-10-16T08:18:51.261904+0000","last_peered":"2023-10-16T08:18:51.261904+0000","last_clean":"2023-10-16T08:18:51.261904+0000","last_became_active":"2023-10-16T08:18:50.268635+0000","last_became_peered":"2023-10-16T08:18:50.268635+0000","last_unstale":"2023-10-16T08:18:51.261904+0000","last_undegraded":"2023-10-16T08:18:51.261904+0000","last_fullsized":"2023-10-16T08:18:51.261904+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:36.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.305941+0000","last_change":"2023-10-16T08:18:49.280831+0000","last_active":"2023-10-16T08:18:50.305941+0000","last_peered":"2023-10-16T08:18:50.305941+0000","last_clean":"2023-10-16T08:18:50.305941+0000","last_became_active":"2023-10-16T08:18:49.280588+0000","last_became_peered":"2023-10-16T08:18:49.280588+0000","last_unstale":"2023-10-16T08:18:50.305941+0000","last_undegraded":"2023-10-16T08:18:50.305941+0000","last_fullsized":"2023-10-16T08:18:50.305941+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:24:55.934837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459427+0000","last_change":"2023-10-16T08:18:50.281051+0000","last_active":"2023-10-16T08:18:52.459427+0000","last_peered":"2023-10-16T08:18:52.459427+0000","last_clean":"2023-10-16T08:18:52.459427+0000","last_became_active":"2023-10-16T08:18:50.280696+0000","last_became_peered":"2023-10-16T08:18:50.280696+0000","last_unstale":"2023-10-16T08:18:52.459427+0000","last_undegraded":"2023-10-16T08:18:52.459427+0000","last_fullsized":"2023-10-16T08:18:52.459427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:13.623970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.259952+0000","last_change":"2023-10-16T08:18:49.280310+0000","last_active":"2023-10-16T08:18:50.259952+0000","last_peered":"2023-10-16T08:18:50.259952+0000","last_clean":"2023-10-16T08:18:50.259952+0000","last_became_active":"2023-10-16T08:18:49.279440+0000","last_became_peered":"2023-10-16T08:18:49.279440+0000","last_unstale":"2023-10-16T08:18:50.259952+0000","last_undegraded":"2023-10-16T08:18:50.259952+0000","last_fullsized":"2023-10-16T08:18:50.259952+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:51.038960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.653443+0000","last_change":"2023-10-16T08:18:50.277009+0000","last_active":"2023-10-16T08:18:51.653443+0000","last_peered":"2023-10-16T08:18:51.653443+0000","last_clean":"2023-10-16T08:18:51.653443+0000","last_became_active":"2023-10-16T08:18:50.276669+0000","last_became_peered":"2023-10-16T08:18:50.276669+0000","last_unstale":"2023-10-16T08:18:51.653443+0000","last_undegraded":"2023-10-16T08:18:51.653443+0000","last_fullsized":"2023-10-16T08:18:51.653443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:02.526716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.262764+0000","last_change":"2023-10-16T08:18:49.275328+0000","last_active":"2023-10-16T08:18:50.262764+0000","last_peered":"2023-10-16T08:18:50.262764+0000","last_clean":"2023-10-16T08:18:50.262764+0000","last_became_active":"2023-10-16T08:18:49.275006+0000","last_became_peered":"2023-10-16T08:18:49.275006+0000","last_unstale":"2023-10-16T08:18:50.262764+0000","last_undegraded":"2023-10-16T08:18:50.262764+0000","last_fullsized":"2023-10-16T08:18:50.262764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:48.800864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459445+0000","last_change":"2023-10-16T08:18:50.295996+0000","last_active":"2023-10-16T08:18:52.459445+0000","last_peered":"2023-10-16T08:18:52.459445+0000","last_clean":"2023-10-16T08:18:52.459445+0000","last_became_active":"2023-10-16T08:18:50.295617+0000","last_became_peered":"2023-10-16T08:18:50.295617+0000","last_unstale":"2023-10-16T08:18:52.459445+0000","last_undegraded":"2023-10-16T08:18:52.459445+0000","last_fullsized":"2023-10-16T08:18:52.459445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:13.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296704+0000","last_change":"2023-10-16T08:18:49.283459+0000","last_active":"2023-10-16T08:18:50.296704+0000","last_peered":"2023-10-16T08:18:50.296704+0000","last_clean":"2023-10-16T08:18:50.296704+0000","last_became_active":"2023-10-16T08:18:49.282767+0000","last_became_peered":"2023-10-16T08:18:49.282767+0000","last_unstale":"2023-10-16T08:18:50.296704+0000","last_undegraded":"2023-10-16T08:18:50.296704+0000","last_fullsized":"2023-10-16T08:18:50.296704+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:08.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459747+0000","last_change":"2023-10-16T08:18:50.282534+0000","last_active":"2023-10-16T08:18:52.459747+0000","last_peered":"2023-10-16T08:18:52.459747+0000","last_clean":"2023-10-16T08:18:52.459747+0000","last_became_active":"2023-10-16T08:18:50.282208+0000","last_became_peered":"2023-10-16T08:18:50.282208+0000","last_unstale":"2023-10-16T08:18:52.459747+0000","last_undegraded":"2023-10-16T08:18:52.459747+0000","last_fullsized":"2023-10-16T08:18:52.459747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:01.834834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.305988+0000","last_change":"2023-10-16T08:18:49.277089+0000","last_active":"2023-10-16T08:18:50.305988+0000","last_peered":"2023-10-16T08:18:50.305988+0000","last_clean":"2023-10-16T08:18:50.305988+0000","last_became_active":"2023-10-16T08:18:49.276783+0000","last_became_peered":"2023-10-16T08:18:49.276783+0000","last_unstale":"2023-10-16T08:18:50.305988+0000","last_undegraded":"2023-10-16T08:18:50.305988+0000","last_fullsized":"2023-10-16T08:18:50.305988+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:42.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261872+0000","last_change":"2023-10-16T08:18:50.271171+0000","last_active":"2023-10-16T08:18:51.261872+0000","last_peered":"2023-10-16T08:18:51.261872+0000","last_clean":"2023-10-16T08:18:51.261872+0000","last_became_active":"2023-10-16T08:18:50.270887+0000","last_became_peered":"2023-10-16T08:18:50.270887+0000","last_unstale":"2023-10-16T08:18:51.261872+0000","last_undegraded":"2023-10-16T08:18:51.261872+0000","last_fullsized":"2023-10-16T08:18:51.261872+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:18.275313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.305345+0000","last_change":"2023-10-16T08:18:49.279592+0000","last_active":"2023-10-16T08:18:50.305345+0000","last_peered":"2023-10-16T08:18:50.305345+0000","last_clean":"2023-10-16T08:18:50.305345+0000","last_became_active":"2023-10-16T08:18:49.279318+0000","last_became_peered":"2023-10-16T08:18:49.279318+0000","last_unstale":"2023-10-16T08:18:50.305345+0000","last_undegraded":"2023-10-16T08:18:50.305345+0000","last_fullsized":"2023-10-16T08:18:50.305345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:39.925292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114895+0000","last_change":"2023-10-16T08:18:50.270888+0000","last_active":"2023-10-16T08:18:52.114895+0000","last_peered":"2023-10-16T08:18:52.114895+0000","last_clean":"2023-10-16T08:18:52.114895+0000","last_became_active":"2023-10-16T08:18:50.270565+0000","last_became_peered":"2023-10-16T08:18:50.270565+0000","last_unstale":"2023-10-16T08:18:52.114895+0000","last_undegraded":"2023-10-16T08:18:52.114895+0000","last_fullsized":"2023-10-16T08:18:52.114895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:18:17.973276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.294474+0000","last_change":"2023-10-16T08:18:49.280302+0000","last_active":"2023-10-16T08:18:50.294474+0000","last_peered":"2023-10-16T08:18:50.294474+0000","last_clean":"2023-10-16T08:18:50.294474+0000","last_became_active":"2023-10-16T08:18:49.280011+0000","last_became_peered":"2023-10-16T08:18:49.280011+0000","last_unstale":"2023-10-16T08:18:50.294474+0000","last_undegraded":"2023-10-16T08:18:50.294474+0000","last_fullsized":"2023-10-16T08:18:50.294474+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:08.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459751+0000","last_change":"2023-10-16T08:18:50.297198+0000","last_active":"2023-10-16T08:18:52.459751+0000","last_peered":"2023-10-16T08:18:52.459751+0000","last_clean":"2023-10-16T08:18:52.459751+0000","last_became_active":"2023-10-16T08:18:50.296966+0000","last_became_peered":"2023-10-16T08:18:50.296966+0000","last_unstale":"2023-10-16T08:18:52.459751+0000","last_undegraded":"2023-10-16T08:18:52.459751+0000","last_fullsized":"2023-10-16T08:18:52.459751+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:54.136989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.261924+0000","last_change":"2023-10-16T08:18:49.275038+0000","last_active":"2023-10-16T08:18:50.261924+0000","last_peered":"2023-10-16T08:18:50.261924+0000","last_clean":"2023-10-16T08:18:50.261924+0000","last_became_active":"2023-10-16T08:18:49.274675+0000","last_became_peered":"2023-10-16T08:18:49.274675+0000","last_unstale":"2023-10-16T08:18:50.261924+0000","last_undegraded":"2023-10-16T08:18:50.261924+0000","last_fullsized":"2023-10-16T08:18:50.261924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.847211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.262036+0000","last_change":"2023-10-16T08:18:50.268973+0000","last_active":"2023-10-16T08:18:51.262036+0000","last_peered":"2023-10-16T08:18:51.262036+0000","last_clean":"2023-10-16T08:18:51.262036+0000","last_became_active":"2023-10-16T08:18:50.268631+0000","last_became_peered":"2023-10-16T08:18:50.268631+0000","last_unstale":"2023-10-16T08:18:51.262036+0000","last_undegraded":"2023-10-16T08:18:51.262036+0000","last_fullsized":"2023-10-16T08:18:51.262036+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:54.136989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.296927+0000","last_change":"2023-10-16T08:18:49.283050+0000","last_active":"2023-10-16T08:18:50.296927+0000","last_peered":"2023-10-16T08:18:50.296927+0000","last_clean":"2023-10-16T08:18:50.296927+0000","last_became_active":"2023-10-16T08:18:49.282767+0000","last_became_peered":"2023-10-16T08:18:49.282767+0000","last_unstale":"2023-10-16T08:18:50.296927+0000","last_undegraded":"2023-10-16T08:18:50.296927+0000","last_fullsized":"2023-10-16T08:18:50.296927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:51.473688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459247+0000","last_change":"2023-10-16T08:18:50.268674+0000","last_active":"2023-10-16T08:18:52.459247+0000","last_peered":"2023-10-16T08:18:52.459247+0000","last_clean":"2023-10-16T08:18:52.459247+0000","last_became_active":"2023-10-16T08:18:50.268413+0000","last_became_peered":"2023-10-16T08:18:50.268413+0000","last_unstale":"2023-10-16T08:18:52.459247+0000","last_undegraded":"2023-10-16T08:18:52.459247+0000","last_fullsized":"2023-10-16T08:18:52.459247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:01.834834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.257440+0000","last_change":"2023-10-16T08:18:49.263362+0000","last_active":"2023-10-16T08:18:50.257440+0000","last_peered":"2023-10-16T08:18:50.257440+0000","last_clean":"2023-10-16T08:18:50.257440+0000","last_became_active":"2023-10-16T08:18:49.262710+0000","last_became_peered":"2023-10-16T08:18:49.262710+0000","last_unstale":"2023-10-16T08:18:50.257440+0000","last_undegraded":"2023-10-16T08:18:50.257440+0000","last_fullsized":"2023-10-16T08:18:50.257440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:51.038960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.458852+0000","last_change":"2023-10-16T08:18:50.279270+0000","last_active":"2023-10-16T08:18:52.458852+0000","last_peered":"2023-10-16T08:18:52.458852+0000","last_clean":"2023-10-16T08:18:52.458852+0000","last_became_active":"2023-10-16T08:18:50.279029+0000","last_became_peered":"2023-10-16T08:18:50.279029+0000","last_unstale":"2023-10-16T08:18:52.458852+0000","last_undegraded":"2023-10-16T08:18:52.458852+0000","last_fullsized":"2023-10-16T08:18:52.458852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:13.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.263877+0000","last_change":"2023-10-16T08:18:49.279019+0000","last_active":"2023-10-16T08:18:50.263877+0000","last_peered":"2023-10-16T08:18:50.263877+0000","last_clean":"2023-10-16T08:18:50.263877+0000","last_became_active":"2023-10-16T08:18:49.278310+0000","last_became_peered":"2023-10-16T08:18:49.278310+0000","last_unstale":"2023-10-16T08:18:50.263877+0000","last_undegraded":"2023-10-16T08:18:50.263877+0000","last_fullsized":"2023-10-16T08:18:50.263877+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:09:59.173567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.261598+0000","last_change":"2023-10-16T08:18:50.279006+0000","last_active":"2023-10-16T08:18:51.261598+0000","last_peered":"2023-10-16T08:18:51.261598+0000","last_clean":"2023-10-16T08:18:51.261598+0000","last_became_active":"2023-10-16T08:18:50.278720+0000","last_became_peered":"2023-10-16T08:18:50.278720+0000","last_unstale":"2023-10-16T08:18:51.261598+0000","last_undegraded":"2023-10-16T08:18:51.261598+0000","last_fullsized":"2023-10-16T08:18:51.261598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:13.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.302476+0000","last_change":"2023-10-16T08:18:49.272412+0000","last_active":"2023-10-16T08:18:50.302476+0000","last_peered":"2023-10-16T08:18:50.302476+0000","last_clean":"2023-10-16T08:18:50.302476+0000","last_became_active":"2023-10-16T08:18:49.272131+0000","last_became_peered":"2023-10-16T08:18:49.272131+0000","last_unstale":"2023-10-16T08:18:50.302476+0000","last_undegraded":"2023-10-16T08:18:50.302476+0000","last_fullsized":"2023-10-16T08:18:50.302476+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:01.519805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.262029+0000","last_change":"2023-10-16T08:18:50.277913+0000","last_active":"2023-10-16T08:18:51.262029+0000","last_peered":"2023-10-16T08:18:51.262029+0000","last_clean":"2023-10-16T08:18:51.262029+0000","last_became_active":"2023-10-16T08:18:50.277628+0000","last_became_peered":"2023-10-16T08:18:50.277628+0000","last_unstale":"2023-10-16T08:18:51.262029+0000","last_undegraded":"2023-10-16T08:18:51.262029+0000","last_fullsized":"2023-10-16T08:18:51.262029+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:01.834834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.261886+0000","last_change":"2023-10-16T08:18:49.273714+0000","last_active":"2023-10-16T08:18:50.261886+0000","last_peered":"2023-10-16T08:18:50.261886+0000","last_clean":"2023-10-16T08:18:50.261886+0000","last_became_active":"2023-10-16T08:18:49.273434+0000","last_became_peered":"2023-10-16T08:18:49.273434+0000","last_unstale":"2023-10-16T08:18:50.261886+0000","last_undegraded":"2023-10-16T08:18:50.261886+0000","last_fullsized":"2023-10-16T08:18:50.261886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:24.205203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259732+0000","last_change":"2023-10-16T08:18:50.271084+0000","last_active":"2023-10-16T08:18:51.259732+0000","last_peered":"2023-10-16T08:18:51.259732+0000","last_clean":"2023-10-16T08:18:51.259732+0000","last_became_active":"2023-10-16T08:18:50.270565+0000","last_became_peered":"2023-10-16T08:18:50.270565+0000","last_unstale":"2023-10-16T08:18:51.259732+0000","last_undegraded":"2023-10-16T08:18:51.259732+0000","last_fullsized":"2023-10-16T08:18:51.259732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:58.648055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.260894+0000","last_change":"2023-10-16T08:18:49.275154+0000","last_active":"2023-10-16T08:18:50.260894+0000","last_peered":"2023-10-16T08:18:50.260894+0000","last_clean":"2023-10-16T08:18:50.260894+0000","last_became_active":"2023-10-16T08:18:49.274869+0000","last_became_peered":"2023-10-16T08:18:49.274869+0000","last_unstale":"2023-10-16T08:18:50.260894+0000","last_undegraded":"2023-10-16T08:18:50.260894+0000","last_fullsized":"2023-10-16T08:18:50.260894+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:12:04.702296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459320+0000","last_change":"2023-10-16T08:18:50.272225+0000","last_active":"2023-10-16T08:18:52.459320+0000","last_peered":"2023-10-16T08:18:52.459320+0000","last_clean":"2023-10-16T08:18:52.459320+0000","last_became_active":"2023-10-16T08:18:50.270658+0000","last_became_peered":"2023-10-16T08:18:50.270658+0000","last_unstale":"2023-10-16T08:18:52.459320+0000","last_undegraded":"2023-10-16T08:18:52.459320+0000","last_fullsized":"2023-10-16T08:18:52.459320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:10.454139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.262661+0000","last_change":"2023-10-16T08:18:49.277635+0000","last_active":"2023-10-16T08:18:50.262661+0000","last_peered":"2023-10-16T08:18:50.262661+0000","last_clean":"2023-10-16T08:18:50.262661+0000","last_became_active":"2023-10-16T08:18:49.277326+0000","last_became_peered":"2023-10-16T08:18:49.277326+0000","last_unstale":"2023-10-16T08:18:50.262661+0000","last_undegraded":"2023-10-16T08:18:50.262661+0000","last_fullsized":"2023-10-16T08:18:50.262661+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.847211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114893+0000","last_change":"2023-10-16T08:18:50.274799+0000","last_active":"2023-10-16T08:18:52.114893+0000","last_peered":"2023-10-16T08:18:52.114893+0000","last_clean":"2023-10-16T08:18:52.114893+0000","last_became_active":"2023-10-16T08:18:50.274546+0000","last_became_peered":"2023-10-16T08:18:50.274546+0000","last_unstale":"2023-10-16T08:18:52.114893+0000","last_undegraded":"2023-10-16T08:18:52.114893+0000","last_fullsized":"2023-10-16T08:18:52.114893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:50.360288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.257233+0000","last_change":"2023-10-16T08:18:49.279346+0000","last_active":"2023-10-16T08:18:50.257233+0000","last_peered":"2023-10-16T08:18:50.257233+0000","last_clean":"2023-10-16T08:18:50.257233+0000","last_became_active":"2023-10-16T08:18:49.279132+0000","last_became_peered":"2023-10-16T08:18:49.279132+0000","last_unstale":"2023-10-16T08:18:50.257233+0000","last_undegraded":"2023-10-16T08:18:50.257233+0000","last_fullsized":"2023-10-16T08:18:50.257233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:24.095898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459541+0000","last_change":"2023-10-16T08:18:50.298066+0000","last_active":"2023-10-16T08:18:52.459541+0000","last_peered":"2023-10-16T08:18:52.459541+0000","last_clean":"2023-10-16T08:18:52.459541+0000","last_became_active":"2023-10-16T08:18:50.297839+0000","last_became_peered":"2023-10-16T08:18:50.297839+0000","last_unstale":"2023-10-16T08:18:52.459541+0000","last_undegraded":"2023-10-16T08:18:52.459541+0000","last_fullsized":"2023-10-16T08:18:52.459541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:08.609731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.267381+0000","last_change":"2023-10-16T08:18:49.274856+0000","last_active":"2023-10-16T08:18:50.267381+0000","last_peered":"2023-10-16T08:18:50.267381+0000","last_clean":"2023-10-16T08:18:50.267381+0000","last_became_active":"2023-10-16T08:18:49.274569+0000","last_became_peered":"2023-10-16T08:18:49.274569+0000","last_unstale":"2023-10-16T08:18:50.267381+0000","last_undegraded":"2023-10-16T08:18:50.267381+0000","last_fullsized":"2023-10-16T08:18:50.267381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:12:04.702296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:51.259333+0000","last_change":"2023-10-16T08:18:50.267604+0000","last_active":"2023-10-16T08:18:51.259333+0000","last_peered":"2023-10-16T08:18:51.259333+0000","last_clean":"2023-10-16T08:18:51.259333+0000","last_became_active":"2023-10-16T08:18:50.267308+0000","last_became_peered":"2023-10-16T08:18:50.267308+0000","last_unstale":"2023-10-16T08:18:51.259333+0000","last_undegraded":"2023-10-16T08:18:51.259333+0000","last_fullsized":"2023-10-16T08:18:51.259333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:48.800013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.257645+0000","last_change":"2023-10-16T08:18:49.270112+0000","last_active":"2023-10-16T08:18:50.257645+0000","last_peered":"2023-10-16T08:18:50.257645+0000","last_clean":"2023-10-16T08:18:50.257645+0000","last_became_active":"2023-10-16T08:18:49.269837+0000","last_became_peered":"2023-10-16T08:18:49.269837+0000","last_unstale":"2023-10-16T08:18:50.257645+0000","last_undegraded":"2023-10-16T08:18:50.257645+0000","last_fullsized":"2023-10-16T08:18:50.257645+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:08.086006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114635+0000","last_change":"2023-10-16T08:18:50.269507+0000","last_active":"2023-10-16T08:18:52.114635+0000","last_peered":"2023-10-16T08:18:52.114635+0000","last_clean":"2023-10-16T08:18:52.114635+0000","last_became_active":"2023-10-16T08:18:50.269221+0000","last_became_peered":"2023-10-16T08:18:50.269221+0000","last_unstale":"2023-10-16T08:18:52.114635+0000","last_undegraded":"2023-10-16T08:18:52.114635+0000","last_fullsized":"2023-10-16T08:18:52.114635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:54:43.295949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.266016+0000","last_change":"2023-10-16T08:18:49.282764+0000","last_active":"2023-10-16T08:18:50.266016+0000","last_peered":"2023-10-16T08:18:50.266016+0000","last_clean":"2023-10-16T08:18:50.266016+0000","last_became_active":"2023-10-16T08:18:49.282469+0000","last_became_peered":"2023-10-16T08:18:49.282469+0000","last_unstale":"2023-10-16T08:18:50.266016+0000","last_undegraded":"2023-10-16T08:18:50.266016+0000","last_fullsized":"2023-10-16T08:18:50.266016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:56:06.397337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.114516+0000","last_change":"2023-10-16T08:18:50.272910+0000","last_active":"2023-10-16T08:18:52.114516+0000","last_peered":"2023-10-16T08:18:52.114516+0000","last_clean":"2023-10-16T08:18:52.114516+0000","last_became_active":"2023-10-16T08:18:50.269982+0000","last_became_peered":"2023-10-16T08:18:50.269982+0000","last_unstale":"2023-10-16T08:18:52.114516+0000","last_undegraded":"2023-10-16T08:18:52.114516+0000","last_fullsized":"2023-10-16T08:18:52.114516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:49.241113+0000","last_clean_scrub_stamp":"2023-10-16T08:18:49.241113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:09.334866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.259321+0000","last_change":"2023-10-16T08:18:49.278621+0000","last_active":"2023-10-16T08:18:50.259321+0000","last_peered":"2023-10-16T08:18:50.259321+0000","last_clean":"2023-10-16T08:18:50.259321+0000","last_became_active":"2023-10-16T08:18:49.278192+0000","last_became_peered":"2023-10-16T08:18:49.278192+0000","last_unstale":"2023-10-16T08:18:50.259321+0000","last_undegraded":"2023-10-16T08:18:50.259321+0000","last_fullsized":"2023-10-16T08:18:50.259321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:18:48.234202+0000","last_clean_scrub_stamp":"2023-10-16T08:18:48.234202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:11.529725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-16T08:18:50.269645+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:18:50.269645+0000","last_peered":"2023-10-16T08:18:50.269645+0000","last_clean":"2023-10-16T08:18:50.269645+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:18:50.269645+0000","last_undegraded":"2023-10-16T08:18:50.269645+0000","last_fullsized":"2023-10-16T08:18:50.269645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1249280,"data_stored":879477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82199999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.48299999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.55500000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.874,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86699999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93200000000000005,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.53800000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91600000000000004,"5min":1.03,"15min":1.03},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97899999999999998,"5min":1.056,"15min":1.056},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95099999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 08:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.57899999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.073,"15min":1.073},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":288949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82899999999999996,"5min":1.077,"15min":1.077},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95399999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72599999999999998},"min":{"1min":0.54200000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.022,"5min":1.343,"15min":1.343},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95499999999999996,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.57899999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.57699999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90400000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.59799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97999999999999998,"5min":1.036,"15min":1.036},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":289601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78100000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.77700000000000002,"5min":2.774,"15min":2.774},"last":0.76700000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84499999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77100000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92500000000000004,"5min":1.099,"15min":1.099},"last":0.88900000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.79600000000000004,"15min":0.82599999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79500000000000004,"5min":2.706,"15min":2.706},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70799999999999996,"15min":0.71799999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.86699999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.85799999999999998,"15min":0.88},"min":{"1min":0.56599999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97199999999999998,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.875,"15min":0.89900000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95899999999999996,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.93700000000000006}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.495},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.55100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55900000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95099999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64300000000000002}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98499999999999999,"5min":1.002,"15min":1.002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.66900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":290561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77600000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75900000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.879}]},{"osd":1,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.82599999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72599999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.80000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85999999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.54500000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91100000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.46899999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84199999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.52300000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.86099999999999999,"5min":1.056,"15min":1.056},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":289005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45900000000000002,"15min":0.46400000000000002},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.46300000000000002,"15min":0.47499999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65900000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.437},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.629}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82599999999999996,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.51800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84899999999999998,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.753},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96099999999999997,"5min":1.355,"15min":1.355},"last":0.80000000000000004}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91900000000000004,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.56100000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92400000000000004,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.85599999999999998,"5min":1.198,"15min":1.198},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.052,"5min":1.325,"15min":1.325},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.82499999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.91400000000000003,"5min":1.391,"15min":1.391},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.91100000000000003,"5min":1.3,"15min":1.3},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":289005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.72299999999999998,"15min":0.77000000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0449999999999999,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.433},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.73299999999999998,"15min":0.78300000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.81599999999999995,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.76500000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.73999999999999999,"15min":0.78700000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":4.399,"15min":4.399},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.74199999999999999,"15min":0.78400000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98999999999999999,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.79700000000000004,"15min":0.84299999999999997},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.383,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.876,"15min":0.92800000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93000000000000005,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.73499999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.0049999999999999,"15min":1.0309999999999999},"min":{"1min":0.501,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5459999999999998,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.006,"15min":1.0309999999999999},"min":{"1min":0.61699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":3.226,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.94799999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0489999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.0249999999999999,"15min":1.0680000000000001},"min":{"1min":0.57199999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.0369999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.93400000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94199999999999995,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.038,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0069999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.90600000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.038,"15min":1.081},"min":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0600000000000001,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":1.0600000000000001,"15min":1.111},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.02,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.878}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":289039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.44500000000000001,"15min":0.45500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64500000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53800000000000003,"5min":1.04,"15min":1.04},"last":0.34300000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.52300000000000002,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.58599999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58899999999999997,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.873,"5min":1.073,"15min":1.073},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59399999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39400000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.84799999999999998},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.82799999999999996,"5min":1.556,"15min":1.556},"last":0.876},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.83099999999999996,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.92300000000000004,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.86199999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.88500000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94399999999999995,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.81999999999999995,"15min":0.83899999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.90100000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86599999999999999,"15min":0.88400000000000001},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.873,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.81200000000000006}]},{"osd":7,"last update":"Mon Oct 16 08:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89000000000000001,"15min":0.91300000000000003},"min":{"1min":0.61399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93899999999999995,"5min":1.913,"15min":1.913},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.89700000000000002,"15min":0.92300000000000004},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92700000000000005,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":881511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.38700000000000001,"15min":0.38},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.44800000000000001,"15min":0.42199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.2999999999999998,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58699999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.56399999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.45200000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.55100000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.78400000000000003,"15min":0.746},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.69399999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.88800000000000001,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:18:54.785 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T08:18:54.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=testcephfs1' mds allow 2023-10-16T08:18:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 Processor -- start 2023-10-16T08:18:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 -- start start 2023-10-16T08:18:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124160e00 0x7f21241656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21240c3cf0 0x7f21240cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2124069e90 0x7f212406a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212406a880 con 0x7f21240c3cf0 2023-10-16T08:18:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240cd7b0 con 0x7f2124069e90 2023-10-16T08:18:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212b89e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240c92c0 con 0x7f2124160e00 2023-10-16T08:18:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21240c3cf0 0x7f21240cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21240c3cf0 0x7f21240cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43586/0 (socket says 172.21.15.32:43586) 2023-10-16T08:18:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 -- 172.21.15.32:0/210689172 learned_addr learned my addr 172.21.15.32:0/210689172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2129e3b640 1 --2- 172.21.15.32:0/210689172 >> v2:172.21.15.186:3300/0 conn(0x7f2124069e90 0x7f212406a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f212963a640 1 --2- 172.21.15.32:0/210689172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124160e00 0x7f21241656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 -- 172.21.15.32:0/210689172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124160e00 msgr2=0x7f21241656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:54.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 --2- 172.21.15.32:0/210689172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124160e00 0x7f21241656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 -- 172.21.15.32:0/210689172 >> v2:172.21.15.186:3300/0 conn(0x7f2124069e90 msgr2=0x7f212406a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:54.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 --2- 172.21.15.32:0/210689172 >> v2:172.21.15.186:3300/0 conn(0x7f2124069e90 0x7f212406a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2128e39640 1 -- 172.21.15.32:0/210689172 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240c9540 con 0x7f21240c3cf0 2023-10-16T08:18:54.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.968+0000 7f2129e3b640 1 --2- 172.21.15.32:0/210689172 >> v2:172.21.15.186:3300/0 conn(0x7f2124069e90 0x7f212406a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:54.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f2128e39640 1 --2- 172.21.15.32:0/210689172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f21240cc9e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f21240cc770 tx=0x7f211401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6ccd03d05db282 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f211bfff640 1 -- 172.21.15.32:0/210689172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140a6040 con 0x7f21240c3cf0 2023-10-16T08:18:54.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f211bfff640 1 -- 172.21.15.32:0/210689172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140aa600 con 0x7f21240c3cf0 2023-10-16T08:18:54.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f211bfff640 1 -- 172.21.15.32:0/210689172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140aa990 con 0x7f21240c3cf0 2023-10-16T08:18:54.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 -- 172.21.15.32:0/210689172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 msgr2=0x7f21240cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 --2- 172.21.15.32:0/210689172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f21240cc9e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f21240cc770 tx=0x7f211401c4e0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 -- 172.21.15.32:0/210689172 shutdown_connections 2023-10-16T08:18:54.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 --2- 172.21.15.32:0/210689172 >> v2:172.21.15.186:3300/0 conn(0x7f2124069e90 0x7f212406a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 --2- 172.21.15.32:0/210689172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f21240cc9e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 --2- 172.21.15.32:0/210689172 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124160e00 0x7f21241656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 -- 172.21.15.32:0/210689172 >> 172.21.15.32:0/210689172 conn(0x7f21240b0d10 msgr2=0x7f21240ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:54.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 -- 172.21.15.32:0/210689172 shutdown_connections 2023-10-16T08:18:54.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.972+0000 7f212b89e640 1 -- 172.21.15.32:0/210689172 wait complete. 2023-10-16T08:18:54.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 Processor -- start 2023-10-16T08:18:54.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- start start 2023-10-16T08:18:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124069e90 0x7f212406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f2124156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2124160e00 0x7f21240c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240c6cc0 con 0x7f21240c3cf0 2023-10-16T08:18:54.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124159db0 con 0x7f2124160e00 2023-10-16T08:18:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212415a090 con 0x7f2124069e90 2023-10-16T08:18:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f2124156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f2124156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43590/0 (socket says 172.21.15.32:43590) 2023-10-16T08:18:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 -- 172.21.15.32:0/292082591 learned_addr learned my addr 172.21.15.32:0/292082591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:54.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2129e3b640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2124160e00 0x7f21240c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212963a640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124069e90 0x7f212406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 -- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124069e90 msgr2=0x7f212406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124069e90 0x7f212406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 -- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2124160e00 msgr2=0x7f21240c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:54.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2124160e00 0x7f21240c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 -- 172.21.15.32:0/292082591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21140094d0 con 0x7f21240c3cf0 2023-10-16T08:18:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2129e3b640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2124160e00 0x7f21240c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2128e39640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f2124156cd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f211401ab90 tx=0x7f211401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140b3800 con 0x7f21240c3cf0 2023-10-16T08:18:54.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140b3960 con 0x7f21240c3cf0 2023-10-16T08:18:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140b3cd0 con 0x7f21240c3cf0 2023-10-16T08:18:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212415a370 con 0x7f21240c3cf0 2023-10-16T08:18:54.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212415c420 con 0x7f21240c3cf0 2023-10-16T08:18:54.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.976+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212404e170 con 0x7f21240c3cf0 2023-10-16T08:18:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.980+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f21140b2380 con 0x7f21240c3cf0 2023-10-16T08:18:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.980+0000 7f2119ffb640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664a0 0x7f2108068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.980+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f211411e610 con 0x7f21240c3cf0 2023-10-16T08:18:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.980+0000 7f212963a640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664a0 0x7f2108068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:54.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.980+0000 7f212963a640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664a0 0x7f2108068960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f212000d8d0 tx=0x7f2120008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:54.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:54.984+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f21140eb9c0 con 0x7f21240c3cf0 2023-10-16T08:18:55.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.200+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]} v 0) v1 -- 0x7f21240c53b0 con 0x7f21240c3cf0 2023-10-16T08:18:55.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f2119ffb640 1 -- 172.21.15.32:0/292082591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 195+0+152 (secure 0 0 0) 0x7f21140eefe0 con 0x7f21240c3cf0 2023-10-16T08:18:55.206 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:18:55.206 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:18:55.206 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-10-16T08:18:55.207 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:18:55.207 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=testcephfs1" 2023-10-16T08:18:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:18:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664a0 msgr2=0x7f2108068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664a0 0x7f2108068960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f212000d8d0 tx=0x7f2120008040 comp rx=0 tx=0).stop 2023-10-16T08:18:55.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 msgr2=0x7f2124156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f2124156cd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f211401ab90 tx=0x7f211401ad20 comp rx=0 tx=0).stop 2023-10-16T08:18:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 shutdown_connections 2023-10-16T08:18:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2124160e00 0x7f21240c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21240c3cf0 0x7f2124156cd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f21080664a0 0x7f2108068960 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 --2- 172.21.15.32:0/292082591 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2124069e90 0x7f212406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 >> 172.21.15.32:0/292082591 conn(0x7f21240b0d10 msgr2=0x7f21240ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:55.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 shutdown_connections 2023-10-16T08:18:55.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.204+0000 7f212b89e640 1 -- 172.21.15.32:0/292082591 wait complete. 2023-10-16T08:18:55.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:18:55.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 Processor -- start 2023-10-16T08:18:55.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 -- start start 2023-10-16T08:18:55.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f131015cb90 0x7f13100cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f13100d3010 0x7f13100d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13100d1400 0x7f13100cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13100d1890 con 0x7f131015cb90 2023-10-16T08:18:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13100ce770 con 0x7f13100d3010 2023-10-16T08:18:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1314d2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13100d0600 con 0x7f13100d1400 2023-10-16T08:18:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f131015cb90 0x7f13100cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f131015cb90 0x7f13100cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43600/0 (socket says 172.21.15.32:43600) 2023-10-16T08:18:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 -- 172.21.15.32:0/2241905001 learned_addr learned my addr 172.21.15.32:0/2241905001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:55.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130ed9d640 1 --2- 172.21.15.32:0/2241905001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13100d1400 0x7f13100cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1307fff640 1 --2- 172.21.15.32:0/2241905001 >> v2:172.21.15.186:3300/0 conn(0x7f13100d3010 0x7f13100d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 -- 172.21.15.32:0/2241905001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13100d1400 msgr2=0x7f13100cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 --2- 172.21.15.32:0/2241905001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13100d1400 0x7f13100cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 -- 172.21.15.32:0/2241905001 >> v2:172.21.15.186:3300/0 conn(0x7f13100d3010 msgr2=0x7f13100d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 --2- 172.21.15.32:0/2241905001 >> v2:172.21.15.186:3300/0 conn(0x7f13100d3010 0x7f13100d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f130e59c640 1 -- 172.21.15.32:0/2241905001 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100c6450 con 0x7f131015cb90 2023-10-16T08:18:55.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.368+0000 7f1307fff640 1 --2- 172.21.15.32:0/2241905001 >> v2:172.21.15.186:3300/0 conn(0x7f13100d3010 0x7f13100d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f130e59c640 1 --2- 172.21.15.32:0/2241905001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131015cb90 0x7f13100cfb10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f131008a880 tx=0x7f130001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b108bec8a62a823d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f130dd9b640 1 -- 172.21.15.32:0/2241905001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000ab600 con 0x7f131015cb90 2023-10-16T08:18:55.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f130dd9b640 1 -- 172.21.15.32:0/2241905001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13000ab760 con 0x7f131015cb90 2023-10-16T08:18:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f130dd9b640 1 -- 172.21.15.32:0/2241905001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000abaf0 con 0x7f131015cb90 2023-10-16T08:18:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 -- 172.21.15.32:0/2241905001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131015cb90 msgr2=0x7f13100cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 --2- 172.21.15.32:0/2241905001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131015cb90 0x7f13100cfb10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f131008a880 tx=0x7f130001c440 comp rx=0 tx=0).stop 2023-10-16T08:18:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 -- 172.21.15.32:0/2241905001 shutdown_connections 2023-10-16T08:18:55.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 --2- 172.21.15.32:0/2241905001 >> v2:172.21.15.186:3300/0 conn(0x7f13100d3010 0x7f13100d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 --2- 172.21.15.32:0/2241905001 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f13100d1400 0x7f13100cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 --2- 172.21.15.32:0/2241905001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131015cb90 0x7f13100cfb10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 -- 172.21.15.32:0/2241905001 >> 172.21.15.32:0/2241905001 conn(0x7f131009bc50 msgr2=0x7f13100d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 -- 172.21.15.32:0/2241905001 shutdown_connections 2023-10-16T08:18:55.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.372+0000 7f1314d2a640 1 -- 172.21.15.32:0/2241905001 wait complete. 2023-10-16T08:18:55.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 Processor -- start 2023-10-16T08:18:55.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 -- start start 2023-10-16T08:18:55.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 0x7f131014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13100d3010 0x7f13100cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131015cb90 0x7f13101f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13101f7a40 con 0x7f13100d1400 2023-10-16T08:18:55.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131014f160 con 0x7f13100d3010 2023-10-16T08:18:55.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13100cba80 con 0x7f131015cb90 2023-10-16T08:18:55.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 0x7f131014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:55.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 0x7f131014ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43602/0 (socket says 172.21.15.32:43602) 2023-10-16T08:18:55.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 -- 172.21.15.32:0/3145108237 learned_addr learned my addr 172.21.15.32:0/3145108237 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:55.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 -- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131015cb90 msgr2=0x7f13101f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131015cb90 0x7f13101f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 -- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13100d3010 msgr2=0x7f13100cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13100d3010 0x7f13100cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 -- 172.21.15.32:0/3145108237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300009480 con 0x7f13100d1400 2023-10-16T08:18:55.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f130e59c640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 0x7f131014ec20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f130001aaf0 tx=0x7f13000191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:55.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000abce0 con 0x7f13100d1400 2023-10-16T08:18:55.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13000abe60 con 0x7f13100d1400 2023-10-16T08:18:55.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300003860 con 0x7f13100d1400 2023-10-16T08:18:55.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13100cb7a0 con 0x7f13100d1400 2023-10-16T08:18:55.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.376+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13101f7660 con 0x7f13100d1400 2023-10-16T08:18:55.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.380+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13101f71e0 con 0x7f13100d1400 2023-10-16T08:18:55.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.380+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13000039c0 con 0x7f13100d1400 2023-10-16T08:18:55.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.380+0000 7f1306ffd640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12f00664f0 0x7f12f00689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:55.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.380+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f130011eaf0 con 0x7f13100d1400 2023-10-16T08:18:55.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.380+0000 7f1307fff640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12f00664f0 0x7f12f00689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:55.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.380+0000 7f1307fff640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12f00664f0 0x7f12f00689b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f13100cade0 tx=0x7f12f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:55.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.384+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f13101f71e0 con 0x7f13100d1400 2023-10-16T08:18:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13101f71e0 con 0x7f13100d1400 2023-10-16T08:18:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1306ffd640 1 -- 172.21.15.32:0/3145108237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f13101f71e0 con 0x7f13100d1400 2023-10-16T08:18:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:55.580 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:18:49.648104+0000","modified":"2023-10-16T08:18:52.672446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/3873719355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3873719355},{"type":"v1","addr":"172.21.15.32:6834","nonce":3873719355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T08:18:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2023-10-16T08:18:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12f00664f0 msgr2=0x7f12f00689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12f00664f0 0x7f12f00689b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f13100cade0 tx=0x7f12f8008040 comp rx=0 tx=0).stop 2023-10-16T08:18:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 msgr2=0x7f131014ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 0x7f131014ec20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f130001aaf0 tx=0x7f13000191e0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 shutdown_connections 2023-10-16T08:18:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12f00664f0 0x7f12f00689b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f131015cb90 0x7f13101f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13100d3010 0x7f13100cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 --2- 172.21.15.32:0/3145108237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13100d1400 0x7f131014ec20 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 >> 172.21.15.32:0/3145108237 conn(0x7f131009bc50 msgr2=0x7f1310158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 shutdown_connections 2023-10-16T08:18:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:55.576+0000 7f1314d2a640 1 -- 172.21.15.32:0/3145108237 wait complete. 2023-10-16T08:18:56.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T08:18:49.648104+0000', 'modified': '2023-10-16T08:18:52.672446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6198}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6198': {'gid': 6198, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6834/3873719355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 3873719355}, {'type': 'v1', 'addr': '172.21.15.32:6834', 'nonce': 3873719355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'testcephfs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T08:18:56.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T08:18:56.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T08:18:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.144+0000 7ffa17765640 1 Processor -- start 2023-10-16T08:18:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa17765640 1 -- start start 2023-10-16T08:18:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa17765640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa1015cf90 0x7ffa1006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa17765640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa17765640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1015d4e0 con 0x7ffa100b2ad0 2023-10-16T08:18:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa17765640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa100ced00 con 0x7ffa1015cf90 2023-10-16T08:18:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa17765640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10160580 con 0x7ffa100ce310 2023-10-16T08:18:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa1015cf90 0x7ffa1006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa1015cf90 0x7ffa1006c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43604/0 (socket says 172.21.15.32:43604) 2023-10-16T08:18:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 -- 172.21.15.32:0/1163285147 learned_addr learned my addr 172.21.15.32:0/1163285147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa15d02640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 -- 172.21.15.32:0/1163285147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 msgr2=0x7ffa100ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 -- 172.21.15.32:0/1163285147 >> v1:172.21.15.186:6789/0 conn(0x7ffa100b2ad0 legacy=0x7ffa1006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa14d00640 1 -- 172.21.15.32:0/1163285147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1015d7c0 con 0x7ffa1015cf90 2023-10-16T08:18:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.148+0000 7ffa15d02640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa14d00640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1015cf90 0x7ffa1006c8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffa000098a0 tx=0x7ffa0001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98f6e1048cbb1ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa07fff640 1 -- 172.21.15.32:0/1163285147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000ab670 con 0x7ffa1015cf90 2023-10-16T08:18:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa07fff640 1 -- 172.21.15.32:0/1163285147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa000ab7d0 con 0x7ffa1015cf90 2023-10-16T08:18:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa07fff640 1 -- 172.21.15.32:0/1163285147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000abb60 con 0x7ffa1015cf90 2023-10-16T08:18:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 -- 172.21.15.32:0/1163285147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1015cf90 msgr2=0x7ffa1006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1015cf90 0x7ffa1006c8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffa000098a0 tx=0x7ffa0001c4b0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 -- 172.21.15.32:0/1163285147 shutdown_connections 2023-10-16T08:18:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 --2- 172.21.15.32:0/1163285147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1015cf90 0x7ffa1006c8f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 -- 172.21.15.32:0/1163285147 >> 172.21.15.32:0/1163285147 conn(0x7ffa100cd1c0 msgr2=0x7ffa100cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 -- 172.21.15.32:0/1163285147 shutdown_connections 2023-10-16T08:18:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 -- 172.21.15.32:0/1163285147 wait complete. 2023-10-16T08:18:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 Processor -- start 2023-10-16T08:18:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.152+0000 7ffa17765640 1 -- start start 2023-10-16T08:18:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 0x7ffa1014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa1015cf90 0x7ffa100b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100b07b0 con 0x7ffa100b2ad0 2023-10-16T08:18:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100b0a90 con 0x7ffa1015cf90 2023-10-16T08:18:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100b0d70 con 0x7ffa100ce310 2023-10-16T08:18:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 0x7ffa1014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 0x7ffa1014cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43620/0 (socket says 172.21.15.32:43620) 2023-10-16T08:18:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 -- 172.21.15.32:0/3276211756 learned_addr learned my addr 172.21.15.32:0/3276211756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa14d00640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15d02640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa1015cf90 0x7ffa100b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 -- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 msgr2=0x7ffa100c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 -- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa1015cf90 msgr2=0x7ffa100b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa1015cf90 0x7ffa100b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 -- 172.21.15.32:0/3276211756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000094d0 con 0x7ffa100b2ad0 2023-10-16T08:18:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15d02640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa1015cf90 0x7ffa100b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:56.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa15501640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 0x7ffa1014cff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00b410 tx=0x7ffa0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa05ffb640 1 -- 172.21.15.32:0/3276211756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004020 con 0x7ffa100b2ad0 2023-10-16T08:18:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa05ffb640 1 -- 172.21.15.32:0/3276211756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c004180 con 0x7ffa100b2ad0 2023-10-16T08:18:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa100b1050 con 0x7ffa100b2ad0 2023-10-16T08:18:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa05ffb640 1 -- 172.21.15.32:0/3276211756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c011630 con 0x7ffa100b2ad0 2023-10-16T08:18:56.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.156+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa100b1610 con 0x7ffa100b2ad0 2023-10-16T08:18:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa05ffb640 1 -- 172.21.15.32:0/3276211756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffa0c011790 con 0x7ffa100b2ad0 2023-10-16T08:18:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa05ffb640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff9f40664a0 0x7ff9f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa05ffb640 1 -- 172.21.15.32:0/3276211756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ffa0c083d00 con 0x7ffa100b2ad0 2023-10-16T08:18:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff9f40664a0 0x7ff9f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 Processor -- start 2023-10-16T08:18:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 -- start start 2023-10-16T08:18:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff9f40664a0 0x7ff9f4068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa000af000 tx=0x7ffa000a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86bcfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86b85710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b86030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a8697acc0 con 0x561a86bd01c0 2023-10-16T08:18:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a869dd920 con 0x561a86bcf8a0 2023-10-16T08:18:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a869dae50 con 0x561a86b85c50 2023-10-16T08:18:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86b85710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86b85710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43626/0 (socket says 172.21.15.32:43626) 2023-10-16T08:18:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 -- 172.21.15.32:0/1549412161 learned_addr learned my addr 172.21.15.32:0/1549412161 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa15501640 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86bcfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa15d02640 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b86030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 -- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 msgr2=0x561a86b86030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b86030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 -- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 msgr2=0x561a86bcfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86bcfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 -- 172.21.15.32:0/1549412161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a866cf890 con 0x561a86bd01c0 2023-10-16T08:18:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa15501640 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86bcfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.160+0000 7ffa14d00640 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86b85710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ffa000ad040 tx=0x7ffa00004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c62fa1fdad6cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ff9faffd640 1 -- 172.21.15.32:0/1549412161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000b6910 con 0x561a86bd01c0 2023-10-16T08:18:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ff9faffd640 1 -- 172.21.15.32:0/1549412161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa000c6070 con 0x561a86bd01c0 2023-10-16T08:18:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ff9faffd640 1 -- 172.21.15.32:0/1549412161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000b6c70 con 0x561a86bd01c0 2023-10-16T08:18:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1549412161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 msgr2=0x561a86b85710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86b85710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ffa000ad040 tx=0x7ffa00004740 comp rx=0 tx=0).stop 2023-10-16T08:18:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1549412161 shutdown_connections 2023-10-16T08:18:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b86030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86b85710 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1549412161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86bcfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1549412161 >> 172.21.15.32:0/1549412161 conn(0x561a86b7cfc0 msgr2=0x561a86b83ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1549412161 shutdown_connections 2023-10-16T08:18:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1549412161 wait complete. 2023-10-16T08:18:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- start start 2023-10-16T08:18:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 Processor -- start 2023-10-16T08:18:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b8aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 0x561a86b8b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bd01c0 0x561a86bbdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a869d6740 con 0x561a86bcf8a0 2023-10-16T08:18:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a86995d10 con 0x561a86bd01c0 2023-10-16T08:18:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a869959b0 con 0x561a86b85c50 2023-10-16T08:18:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa1aaf1000 10 client.0 _ll_register_callbacks cb 0x561a86b71390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:18:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 0x561a86b8b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 0x561a86b8b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43628/0 (socket says 172.21.15.32:43628) 2023-10-16T08:18:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 -- 172.21.15.32:0/3466247039 learned_addr learned my addr 172.21.15.32:0/3466247039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa15d02640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bd01c0 0x561a86bbdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa15501640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b8aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 -- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 msgr2=0x561a86b8aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b8aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 -- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bd01c0 msgr2=0x561a86bbdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bd01c0 0x561a86bbdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 -- 172.21.15.32:0/3466247039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa000bc070 con 0x561a86bcf8a0 2023-10-16T08:18:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa15d02640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bd01c0 0x561a86bbdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ffa14d00640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 0x561a86b8b420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ffa0001cd50 tx=0x7ffa0001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed92284a76f81bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.164+0000 7ff9effff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:18:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 1 -- 172.21.15.32:0/3466247039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000b6c90 con 0x561a86bcf8a0 2023-10-16T08:18:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a8698a370 con 0x561a86bcf8a0 2023-10-16T08:18:56.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 1 -- 172.21.15.32:0/3466247039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ffa000b6df0 con 0x561a86bcf8a0 2023-10-16T08:18:56.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 1 -- 172.21.15.32:0/3466247039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa000cb430 con 0x561a86bcf8a0 2023-10-16T08:18:56.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 10 client.6336 fetch_fsmap learned FSMap version 28 2023-10-16T08:18:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x561a86998460 con 0x561a86bcf8a0 2023-10-16T08:18:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 1 -- 172.21.15.32:0/3466247039 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a8698a370 con 0x561a86bcf8a0 2023-10-16T08:18:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 1 -- 172.21.15.32:0/3466247039 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 3299+0+0 (secure 0 0 0) 0x7ffa000cb590 con 0x561a86bcf8a0 2023-10-16T08:18:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 10 client.6336 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T08:18:56.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 10 client.6336 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T08:18:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 20 client.6336 populate_metadata read hostname 'smithi032' 2023-10-16T08:18:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86bdf940 0x561a86be1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 4 client.6336 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T08:18:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 --> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a869702d0 con 0x561a86bdf940 2023-10-16T08:18:56.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa15501640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86bdf940 0x561a86be1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa15501640 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86bdf940 0x561a86be1d30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c3c0c100393643 server_cookie=4b7e024b4d9a5eb6 in_seq=0 out_seq=0 2023-10-16T08:18:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 10 client.6336 ms_handle_connect on v2:172.21.15.32:6833/3873719355 2023-10-16T08:18:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 1 -- 172.21.15.32:0/3466247039 <== mds.0 v2:172.21.15.32:6833/3873719355 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a869702d0 con 0x561a86bdf940 2023-10-16T08:18:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 10 client.6336 handle_command_reply: tid=0 2023-10-16T08:18:56.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 client.6336 shutdown 2023-10-16T08:18:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 msgr2=0x561a86b8b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 0x561a86b8b420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ffa0001cd50 tx=0x7ffa0001acf0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 shutdown_connections 2023-10-16T08:18:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86bdf940 0x561a86be1d30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bd01c0 0x561a86bbdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bcf8a0 0x561a86b8b420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/3466247039 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86b8aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ff9effff640 0 client.6336 ms_handle_reset on v2:172.21.15.32:6833/3873719355 2023-10-16T08:18:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 >> 172.21.15.32:0/3466247039 conn(0x561a86b7cfc0 msgr2=0x561a86b846f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 shutdown_connections 2023-10-16T08:18:56.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/3466247039 wait complete. 2023-10-16T08:18:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.168+0000 7ffa1aaf1000 20 client.6336 trim_cache size 0 max 16384 2023-10-16T08:18:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 Processor -- start 2023-10-16T08:18:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 -- start start 2023-10-16T08:18:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86be2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86be3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86bb89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a86690940 con 0x561a86bd01c0 2023-10-16T08:18:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a869846c0 con 0x561a86bcf8a0 2023-10-16T08:18:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a869aca10 con 0x561a86b85c50 2023-10-16T08:18:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86bb89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86bb89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43632/0 (socket says 172.21.15.32:43632) 2023-10-16T08:18:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 -- 172.21.15.32:0/1356921445 learned_addr learned my addr 172.21.15.32:0/1356921445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 -- 172.21.15.32:0/1356921445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 msgr2=0x561a86be2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86be2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 -- 172.21.15.32:0/1356921445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 msgr2=0x561a86be3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86be3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.184+0000 7ffa15d02640 1 -- 172.21.15.32:0/1356921445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a8697e760 con 0x561a86bd01c0 2023-10-16T08:18:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15d02640 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86bb89a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ffa08010430 tx=0x7ffa080108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00fc154747ee1b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ff9fb7fe640 1 -- 172.21.15.32:0/1356921445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08015070 con 0x561a86bd01c0 2023-10-16T08:18:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ff9fb7fe640 1 -- 172.21.15.32:0/1356921445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa080039a0 con 0x561a86bd01c0 2023-10-16T08:18:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ff9fb7fe640 1 -- 172.21.15.32:0/1356921445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08003d10 con 0x561a86bd01c0 2023-10-16T08:18:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1356921445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 msgr2=0x561a86bb89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86bb89a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ffa08010430 tx=0x7ffa080108f0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1356921445 shutdown_connections 2023-10-16T08:18:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86bd01c0 0x561a86bb89a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86be3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/1356921445 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86b85c50 0x561a86be2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1356921445 >> 172.21.15.32:0/1356921445 conn(0x561a86b7cfc0 msgr2=0x561a86bdce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1356921445 shutdown_connections 2023-10-16T08:18:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/1356921445 wait complete. 2023-10-16T08:18:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- start start 2023-10-16T08:18:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 Processor -- start 2023-10-16T08:18:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 0x561a86c2e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86c2e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86bd01c0 0x561a86bbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a86b8a6a0 con 0x561a86b85c50 2023-10-16T08:18:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a869d0a40 con 0x561a86bcf8a0 2023-10-16T08:18:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a86a17ba0 con 0x561a86bd01c0 2023-10-16T08:18:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa1aaf1000 10 client.0 _ll_register_callbacks cb 0x7ffa000ab470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:18:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 0x561a86c2e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 0x561a86c2e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43636/0 (socket says 172.21.15.32:43636) 2023-10-16T08:18:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 -- 172.21.15.32:0/4004087307 learned_addr learned my addr 172.21.15.32:0/4004087307 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa14d00640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86c2e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15d02640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86bd01c0 0x561a86bbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 -- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86bd01c0 msgr2=0x561a86bbb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86bd01c0 0x561a86bbb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 -- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 msgr2=0x561a86c2e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86c2e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 -- 172.21.15.32:0/4004087307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa08007ca0 con 0x561a86b85c50 2023-10-16T08:18:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ffa15501640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 0x561a86c2e290 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0091f0 tx=0x7ffa0c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d8c822a7168f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.188+0000 7ff9effff640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-10-16T08:18:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 1 -- 172.21.15.32:0/4004087307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c071070 con 0x561a86b85c50 2023-10-16T08:18:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a8696e6f0 con 0x561a86b85c50 2023-10-16T08:18:56.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 1 -- 172.21.15.32:0/4004087307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ffa0c052fe0 con 0x561a86b85c50 2023-10-16T08:18:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 1 -- 172.21.15.32:0/4004087307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa0c051350 con 0x561a86b85c50 2023-10-16T08:18:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 10 client.6342 fetch_fsmap learned FSMap version 28 2023-10-16T08:18:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x561a866cfcc0 con 0x561a86b85c50 2023-10-16T08:18:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 1 -- 172.21.15.32:0/4004087307 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a8696e6f0 con 0x561a86b85c50 2023-10-16T08:18:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 1 -- 172.21.15.32:0/4004087307 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 3299+0+0 (secure 0 0 0) 0x7ffa0c05cab0 con 0x561a86b85c50 2023-10-16T08:18:56.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 10 client.6342 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T08:18:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 10 client.6342 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T08:18:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 20 client.6342 populate_metadata read hostname 'smithi032' 2023-10-16T08:18:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86c35160 0x561a86c37550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 4 client.6342 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T08:18:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 --> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561a869cf720 con 0x561a86c35160 2023-10-16T08:18:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa14d00640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86c35160 0x561a86c37550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa14d00640 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86c35160 0x561a86c37550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65aa3d41008fa851 server_cookie=a5718512e9eba5bd in_seq=0 out_seq=0 2023-10-16T08:18:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 10 client.6342 ms_handle_connect on v2:172.21.15.32:6833/3873719355 2023-10-16T08:18:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 1 -- 172.21.15.32:0/4004087307 <== mds.0 v2:172.21.15.32:6833/3873719355 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x561a869cf720 con 0x561a86c35160 2023-10-16T08:18:56.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 10 client.6342 handle_command_reply: tid=0 2023-10-16T08:18:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 client.6342 shutdown 2023-10-16T08:18:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 msgr2=0x561a86c2e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 0x561a86c2e290 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0091f0 tx=0x7ffa0c004690 comp rx=0 tx=0).stop 2023-10-16T08:18:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 shutdown_connections 2023-10-16T08:18:56.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:6833/3873719355,v1:172.21.15.32:6834/3873719355] conn(0x561a86c35160 0x561a86c37550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x561a86bd01c0 0x561a86bbb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561a86bcf8a0 0x561a86c2e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 --2- 172.21.15.32:0/4004087307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561a86b85c50 0x561a86c2e290 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ff9effff640 0 client.6342 ms_handle_reset on v2:172.21.15.32:6833/3873719355 2023-10-16T08:18:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 >> 172.21.15.32:0/4004087307 conn(0x561a86b7cfc0 msgr2=0x561a86be1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 shutdown_connections 2023-10-16T08:18:56.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 1 -- 172.21.15.32:0/4004087307 wait complete. 2023-10-16T08:18:56.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa1aaf1000 20 client.6342 trim_cache size 0 max 16384 2023-10-16T08:18:56.210 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:18:56.211 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "943c836a-5c4c-4cb7-a190-9643b3476f99", 2023-10-16T08:18:56.211 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-10-16T08:18:56.211 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6198, 2023-10-16T08:18:56.211 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-10-16T08:18:56.211 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-10-16T08:18:56.212 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1", 2023-10-16T08:18:56.212 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9152849590000001, 2023-10-16T08:18:56.212 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 28, 2023-10-16T08:18:56.212 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 54, 2023-10-16T08:18:56.212 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 54, 2023-10-16T08:18:56.212 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.814210478 2023-10-16T08:18:56.213 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:18:56.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff9f40664a0 msgr2=0x7ff9f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff9f40664a0 0x7ff9f4068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa000af000 tx=0x7ffa000a7000 comp rx=0 tx=0).stop 2023-10-16T08:18:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 msgr2=0x7ffa1014cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 0x7ffa1014cff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00b410 tx=0x7ffa0c00b8d0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 shutdown_connections 2023-10-16T08:18:56.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa1015cf90 0x7ffa100b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff9f40664a0 0x7ff9f4068960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ffa100ce310 0x7ffa100c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 --2- 172.21.15.32:0/3276211756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa100b2ad0 0x7ffa1014cff0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 >> 172.21.15.32:0/3276211756 conn(0x7ffa100cd1c0 msgr2=0x7ffa10072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 shutdown_connections 2023-10-16T08:18:56.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.192+0000 7ffa17765640 1 -- 172.21.15.32:0/3276211756 wait complete. 2023-10-16T08:18:56.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:18:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.356+0000 7ff30946b640 1 Processor -- start 2023-10-16T08:18:56.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff30946b640 1 -- start start 2023-10-16T08:18:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff30946b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff30415ca70 0x7ff3040d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff30946b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040c65f0 con 0x7ff3040d3550 2023-10-16T08:18:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff30946b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040cdf80 con 0x7ff3040cf7a0 2023-10-16T08:18:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff30946b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3040ce820 con 0x7ff30415ca70 2023-10-16T08:18:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff2fbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3040d3550 0x7ff3040cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42108/0 (socket says 172.21.15.32:42108) 2023-10-16T08:18:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff2fbfff640 1 -- 172.21.15.32:0/2919995667 learned_addr learned my addr 172.21.15.32:0/2919995667 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff302ffd640 1 --2- 172.21.15.32:0/2919995667 >> v2:172.21.15.186:3300/0 conn(0x7ff30415ca70 0x7ff3040d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458977007 0 0) 0x7ff3040c65f0 con 0x7ff3040d3550 2023-10-16T08:18:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f0003530 con 0x7ff3040d3550 2023-10-16T08:18:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2490829202 0 0) 0x7ff2f0003530 con 0x7ff3040d3550 2023-10-16T08:18:56.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 >> v2:172.21.15.186:3300/0 conn(0x7ff30415ca70 msgr2=0x7ff3040d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 --2- 172.21.15.32:0/2919995667 >> v2:172.21.15.186:3300/0 conn(0x7ff30415ca70 0x7ff3040d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 >> v1:172.21.15.186:6789/0 conn(0x7ff3040cf7a0 legacy=0x7ff3040c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040ce260 con 0x7ff3040d3550 2023-10-16T08:18:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7ff2ec001680 con 0x7ff3040d3550 2023-10-16T08:18:56.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 >> v1:172.21.15.32:6789/0 conn(0x7ff3040d3550 legacy=0x7ff3040cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f0003980 0x7ff2f0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0004380 0x7ff2f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f0005050 0x7ff2f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040ce820 con 0x7ff2f0005050 2023-10-16T08:18:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f000bfb0 con 0x7ff2f0004380 2023-10-16T08:18:56.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f000c170 con 0x7ff2f0003980 2023-10-16T08:18:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3037fe640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f0005050 0x7ff2f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff2fbfff640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0004380 0x7ff2f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff302ffd640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f0003980 0x7ff2f0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3037fe640 1 -- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f0003980 msgr2=0x7ff2f0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3037fe640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff2f0003980 0x7ff2f0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3037fe640 1 -- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0004380 msgr2=0x7ff2f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3037fe640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0004380 0x7ff2f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.360+0000 7ff3037fe640 1 -- 172.21.15.32:0/2919995667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0003530 con 0x7ff2f0005050 2023-10-16T08:18:56.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff3037fe640 1 -- 172.21.15.32:0/2919995667 reap_dead start 2023-10-16T08:18:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff3037fe640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f0005050 0x7ff2f000b990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff3041605d0 tx=0x7ff2fc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7d1184f01fd906 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec007ea0 con 0x7ff2f0005050 2023-10-16T08:18:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc003ba0 con 0x7ff2f0005050 2023-10-16T08:18:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff3027fc640 1 -- 172.21.15.32:0/2919995667 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0050d0 con 0x7ff2f0005050 2023-10-16T08:18:56.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 -- 172.21.15.32:0/2919995667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f0005050 msgr2=0x7ff2f000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f0005050 0x7ff2f000b990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff3041605d0 tx=0x7ff2fc01c8a0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 -- 172.21.15.32:0/2919995667 shutdown_connections 2023-10-16T08:18:56.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 --2- 172.21.15.32:0/2919995667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2f0005050 0x7ff2f000b990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 -- 172.21.15.32:0/2919995667 >> 172.21.15.32:0/2919995667 conn(0x7ff30409bc50 msgr2=0x7ff3040d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 -- 172.21.15.32:0/2919995667 shutdown_connections 2023-10-16T08:18:56.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.364+0000 7ff30946b640 1 -- 172.21.15.32:0/2919995667 wait complete. 2023-10-16T08:18:56.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 Processor -- start 2023-10-16T08:18:56.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- start start 2023-10-16T08:18:56.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0005050 0x7ff3041f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 0x7ff3040c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3040c6fa0 0x7ff3040c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c7420 con 0x7ff30415b3e0 2023-10-16T08:18:56.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30415af50 con 0x7ff2f0005050 2023-10-16T08:18:56.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30415a620 con 0x7ff3040c6fa0 2023-10-16T08:18:56.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 0x7ff3040c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 0x7ff3040c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43648/0 (socket says 172.21.15.32:43648) 2023-10-16T08:18:56.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 -- 172.21.15.32:0/462960402 learned_addr learned my addr 172.21.15.32:0/462960402 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:56.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff302ffd640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0005050 0x7ff3041f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff3037fe640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3040c6fa0 0x7ff3040c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 -- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3040c6fa0 msgr2=0x7ff3040c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3040c6fa0 0x7ff3040c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 -- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0005050 msgr2=0x7ff3041f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0005050 0x7ff3041f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 -- 172.21.15.32:0/462960402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30415a900 con 0x7ff30415b3e0 2023-10-16T08:18:56.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff3037fe640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3040c6fa0 0x7ff3040c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:56.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fbfff640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 0x7ff3040c6a60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0124c0 tx=0x7ff2ec0128c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec014750 con 0x7ff30415b3e0 2023-10-16T08:18:56.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec0148d0 con 0x7ff30415b3e0 2023-10-16T08:18:56.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec014c80 con 0x7ff30415b3e0 2023-10-16T08:18:56.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30415abe0 con 0x7ff30415b3e0 2023-10-16T08:18:56.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3041f1df0 con 0x7ff30415b3e0 2023-10-16T08:18:56.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.368+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff304074010 con 0x7ff30415b3e0 2023-10-16T08:18:56.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.372+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff2ec0153e0 con 0x7ff30415b3e0 2023-10-16T08:18:56.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.372+0000 7ff2fb7fe640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2e40664d0 0x7ff2e4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:56.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.372+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ff2ec086610 con 0x7ff30415b3e0 2023-10-16T08:18:56.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.372+0000 7ff302ffd640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2e40664d0 0x7ff2e4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:56.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.372+0000 7ff302ffd640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2e40664d0 0x7ff2e4068990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009b40 tx=0x7ff2f4006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:56.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.376+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff2ec053930 con 0x7ff30415b3e0 2023-10-16T08:18:56.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.564+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3040b85b0 con 0x7ff30415b3e0 2023-10-16T08:18:56.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.564+0000 7ff2fb7fe640 1 -- 172.21.15.32:0/462960402 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff2ec056f50 con 0x7ff30415b3e0 2023-10-16T08:18:56.569 INFO:teuthology.orchestra.run.smithi032.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T08:18:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2e40664d0 msgr2=0x7ff2e4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2e40664d0 0x7ff2e4068990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009b40 tx=0x7ff2f4006f90 comp rx=0 tx=0).stop 2023-10-16T08:18:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 msgr2=0x7ff3040c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 0x7ff3040c6a60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0124c0 tx=0x7ff2ec0128c0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 shutdown_connections 2023-10-16T08:18:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff2e40664d0 0x7ff2e4068990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff3040c6fa0 0x7ff3040c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff30415b3e0 0x7ff3040c6a60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 --2- 172.21.15.32:0/462960402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0005050 0x7ff3041f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 >> 172.21.15.32:0/462960402 conn(0x7ff30409bc50 msgr2=0x7ff3040b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 shutdown_connections 2023-10-16T08:18:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:56.568+0000 7ff30946b640 1 -- 172.21.15.32:0/462960402 wait complete. 2023-10-16T08:18:56.589 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:18:56.590 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:18:56.590 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:18:56.591 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:18:56.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:56.591 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:18:56.591 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:18:56.592 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:18:56.592 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:56.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:56.593 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:18:56.603 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:18:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:18:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:18:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:18:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.604 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:56.604 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:18:56.605 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:56.605 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:18:56.606 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:18:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:18:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:18:56.606 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42268sec preferred_lft 42268sec 2023-10-16T08:18:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:18:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.607 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:18:56.607 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:18:56.608 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:18:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:56.608 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:18:56.609 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.609 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:18:56.609 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:18:56.609 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:18:56.609 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.610 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:18:56.610 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.610 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:18:56.610 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:56.610 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:18:56.611 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.611 INFO:teuthology.orchestra.run.smithi032.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:18:56.611 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:56.611 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T08:18:56.611 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:18:56.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:56.612 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:18:56.664 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:18:56.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:56.667 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:18:56.718 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:18:56.719 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:18:56.719 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:18:56.719 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:18:56.719 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:18:56.720 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:18:56.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:56.720 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:18:56.721 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:18:56.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:18:56.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:18:56.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:18:56.722 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:18:56.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:56.813 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:18:56.823 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:18:56.824 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:18:56.824 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:56.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:56.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:18:56.885 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:18:56.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:18:56.886 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:56.937 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:18:56.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:18:56.938 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:56.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:18:57.059 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:18:57.111 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:18:57.111 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:18:57.111 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:18:57.111 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:18:57.112 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:18:57.113 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:18:57.113 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:18:57.113 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:18:57.113 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:18:57.114 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:18:57.114 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:18:57.114 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:18:57.114 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:18:57.114 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:18:57.115 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:18:57.115 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:18:57.115 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:18:57.115 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:18:57.115 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:18:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:18:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:18:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:18:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:18:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:18:57.116 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:18:57.117 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:57.117 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:57.117 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:57.117 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:57.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:57.119 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:18:57.169 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:18:57.169 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:18:57.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:18:57.239 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:18:57.291 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:18:57.291 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:18:57.291 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:18:57.292 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:18:57.292 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:18:57.292 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:18:57.292 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:18:57.292 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:18:57.292 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:18:57.293 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:18:57.293 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:18:57.293 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:18:57.293 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:18:57.293 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:18:57.294 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:18:57.294 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:18:57.294 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:18:57.294 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:18:57.294 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:18:57.294 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:18:57.295 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:18:57.295 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:18:57.295 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:18:57.295 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:18:57.295 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:18:57.296 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:18:57.296 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:18:57.296 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:18:57.296 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:18:57.296 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:57.297 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:18:57.297 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:57.297 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:18:57.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:57.298 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:18:57.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:18:57.312+0000 7ff129a2a580 -1 init, newargv = 0x55db60c56ce0 newargc=15 2023-10-16T08:18:57.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[50936]: starting ceph client 2023-10-16T08:18:57.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[50936]: starting fuse 2023-10-16T08:18:57.324 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:18:57.325 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:18:57.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:18:57.326 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:18:57.326 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:18:57.326 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:18:57.327 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:18:57.327 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:57.327 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:18:57.328 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:18:57.328 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:18:57.329 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:18:57.329 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:57.329 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:18:57.330 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:18:57.330 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:18:57.330 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:18:57.331 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:57.331 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:18:57.331 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:18:57.332 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:18:57.332 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:18:57.333 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:18:57.333 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:18:57.333 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:18:57.334 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:18:57.334 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:18:57.334 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:18:57.335 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:18:57.456 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.50936.asok 2023-10-16T08:18:57.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:18:57.467 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50936.asok 2023-10-16T08:18:57.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:57.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50936.asok status 2023-10-16T08:18:57.643 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:18:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:18:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:18:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:18:57.644 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:18:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:18:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:18:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "50936", 2023-10-16T08:18:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:18:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:57.645 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:18:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:18:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6354, 2023-10-16T08:18:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:18:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:18:57.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:18:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 6354 2023-10-16T08:18:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:18:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:18:57.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:18:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2242440413 2023-10-16T08:18:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:18:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:18:57.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:18:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:18:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2242440413 2023-10-16T08:18:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:18:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.6354 192.168.0.1:0/2242440413", 2023-10-16T08:18:57.649 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/2242440413", 2023-10-16T08:18:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:18:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 28, 2023-10-16T08:18:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 54, 2023-10-16T08:18:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:18:57.650 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:18:57.651 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1" 2023-10-16T08:18:57.651 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:18:57.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:57.659 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:18:57.660 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:18:57.669 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:57.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:57.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:18:57.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:18:57.734 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-10-16T08:18:57.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=testcephfs1_data' 2023-10-16T08:18:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fbe17640 1 Processor -- start 2023-10-16T08:18:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fbe17640 1 -- start start 2023-10-16T08:18:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fbe17640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12f406b6b0 0x7f12f41648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fbe17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f40cb0f0 con 0x7f12f40aedf0 2023-10-16T08:18:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fbe17640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f40cb320 con 0x7f12f4159900 2023-10-16T08:18:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fbe17640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f40cb600 con 0x7f12f406b6b0 2023-10-16T08:18:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fa3b4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12f406b6b0 0x7f12f41648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12f93b2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f12f40aedf0 0x7f12f406afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42110/0 (socket says 172.21.15.32:42110) 2023-10-16T08:18:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12f93b2640 1 -- 172.21.15.32:0/1534317891 learned_addr learned my addr 172.21.15.32:0/1534317891 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fa3b4640 1 -- 172.21.15.32:0/1534317891 >> v1:172.21.15.186:6789/0 conn(0x7f12f4159900 legacy=0x7f12f40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fa3b4640 1 -- 172.21.15.32:0/1534317891 >> v1:172.21.15.32:6789/0 conn(0x7f12f40aedf0 legacy=0x7f12f406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12fa3b4640 1 -- 172.21.15.32:0/1534317891 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f40cb8e0 con 0x7f12f406b6b0 2023-10-16T08:18:57.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.980+0000 7f12f8bb1640 1 -- 172.21.15.32:0/1534317891 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394441900 0 0) 0x7f12f40cb0f0 con 0x7f12f40aedf0 2023-10-16T08:18:57.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fa3b4640 1 --2- 172.21.15.32:0/1534317891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f41648c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009d90 tx=0x7f12ec01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23935b42b07559f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12f8bb1640 1 -- 172.21.15.32:0/1534317891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0a59e0 con 0x7f12f406b6b0 2023-10-16T08:18:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12f8bb1640 1 -- 172.21.15.32:0/1534317891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec0a5b40 con 0x7f12f406b6b0 2023-10-16T08:18:57.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12f8bb1640 1 -- 172.21.15.32:0/1534317891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0ad440 con 0x7f12f406b6b0 2023-10-16T08:18:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 -- 172.21.15.32:0/1534317891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 msgr2=0x7f12f41648c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 --2- 172.21.15.32:0/1534317891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f41648c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009d90 tx=0x7f12ec01c920 comp rx=0 tx=0).stop 2023-10-16T08:18:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 -- 172.21.15.32:0/1534317891 shutdown_connections 2023-10-16T08:18:57.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 --2- 172.21.15.32:0/1534317891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f41648c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 -- 172.21.15.32:0/1534317891 >> 172.21.15.32:0/1534317891 conn(0x7f12f40cf820 msgr2=0x7f12f40cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 -- 172.21.15.32:0/1534317891 shutdown_connections 2023-10-16T08:18:57.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.984+0000 7f12fbe17640 1 -- 172.21.15.32:0/1534317891 wait complete. 2023-10-16T08:18:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 Processor -- start 2023-10-16T08:18:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- start start 2023-10-16T08:18:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f40b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f40aedf0 0x7f12f41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12f4159900 0x7f12f40c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40b10d0 con 0x7f12f406b6b0 2023-10-16T08:18:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40b3f80 con 0x7f12f40aedf0 2023-10-16T08:18:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40b4260 con 0x7f12f4159900 2023-10-16T08:18:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f40b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f40b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43666/0 (socket says 172.21.15.32:43666) 2023-10-16T08:18:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 -- 172.21.15.32:0/3485480972 learned_addr learned my addr 172.21.15.32:0/3485480972 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f93b2640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f40aedf0 0x7f12f41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fa3b4640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12f4159900 0x7f12f40c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 -- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12f4159900 msgr2=0x7f12f40c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:57.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12f4159900 0x7f12f40c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 -- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f40aedf0 msgr2=0x7f12f41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f40aedf0 0x7f12f41580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:57.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 -- 172.21.15.32:0/3485480972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec009a70 con 0x7f12f406b6b0 2023-10-16T08:18:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12f9bb3640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f40b0a40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c3c0 tx=0x7f12f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00052d0 con 0x7f12f406b6b0 2023-10-16T08:18:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0005430 con 0x7f12f406b6b0 2023-10-16T08:18:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0003840 con 0x7f12f406b6b0 2023-10-16T08:18:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f40b4540 con 0x7f12f406b6b0 2023-10-16T08:18:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f4159180 con 0x7f12f406b6b0 2023-10-16T08:18:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.988+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f40ca4a0 con 0x7f12f406b6b0 2023-10-16T08:18:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.992+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f12f00039a0 con 0x7f12f406b6b0 2023-10-16T08:18:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.992+0000 7f12ea7fc640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12d80664a0 0x7f12d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.992+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f12f0083520 con 0x7f12f406b6b0 2023-10-16T08:18:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.992+0000 7f12f93b2640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12d80664a0 0x7f12d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.992+0000 7f12f93b2640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12d80664a0 0x7f12d8068960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12f40d00e0 tx=0x7f12e4007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:57.996+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f12f0050850 con 0x7f12f406b6b0 2023-10-16T08:18:58.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.208+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=testcephfs1_data"]} v 0) v1 -- 0x7f12f404e170 con 0x7f12f406b6b0 2023-10-16T08:18:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12ea7fc640 1 -- 172.21.15.32:0/3485480972 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=testcephfs1_data"]}]=0 updated caps for client.0 v14) v1 ==== 188+0+145 (secure 0 0 0) 0x7f12f0053e70 con 0x7f12f406b6b0 2023-10-16T08:18:58.213 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:18:58.214 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:18:58.214 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-10-16T08:18:58.214 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:18:58.214 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow r pool=testcephfs1_data" 2023-10-16T08:18:58.215 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:18:58.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12d80664a0 msgr2=0x7f12d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12d80664a0 0x7f12d8068960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12f40d00e0 tx=0x7f12e4007240 comp rx=0 tx=0).stop 2023-10-16T08:18:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 msgr2=0x7f12f40b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f40b0a40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c3c0 tx=0x7f12f000c880 comp rx=0 tx=0).stop 2023-10-16T08:18:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 shutdown_connections 2023-10-16T08:18:58.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f12d80664a0 0x7f12d8068960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f12f4159900 0x7f12f40c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12f40aedf0 0x7f12f41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 --2- 172.21.15.32:0/3485480972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12f406b6b0 0x7f12f40b0a40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 >> 172.21.15.32:0/3485480972 conn(0x7f12f40cf820 msgr2=0x7f12f4163ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:58.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 shutdown_connections 2023-10-16T08:18:58.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.212+0000 7f12fbe17640 1 -- 172.21.15.32:0/3485480972 wait complete. 2023-10-16T08:18:58.235 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:18:58.236 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:18:58.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:18:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 Processor -- start 2023-10-16T08:18:58.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 -- start start 2023-10-16T08:18:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39540d15d0 0x7f39540cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d3010 0x7f39540c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39540ce770 con 0x7f395415cb90 2023-10-16T08:18:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39540d0600 con 0x7f39540d15d0 2023-10-16T08:18:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3959c32640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540c6450 con 0x7f39540d3010 2023-10-16T08:18:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39540d15d0 0x7f39540cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39540d15d0 0x7f39540cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43678/0 (socket says 172.21.15.32:43678) 2023-10-16T08:18:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 -- 172.21.15.32:0/1191195559 learned_addr learned my addr 172.21.15.32:0/1191195559 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f3953fff640 1 --2- 172.21.15.32:0/1191195559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d3010 0x7f39540c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 -- 172.21.15.32:0/1191195559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d3010 msgr2=0x7f39540c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:58.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 --2- 172.21.15.32:0/1191195559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d3010 0x7f39540c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 -- 172.21.15.32:0/1191195559 >> v1:172.21.15.186:6789/0 conn(0x7f395415cb90 legacy=0x7f39540d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.432+0000 7f394bfff640 1 -- 172.21.15.32:0/1191195559 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39540d08a0 con 0x7f39540d15d0 2023-10-16T08:18:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f394bfff640 1 --2- 172.21.15.32:0/1191195559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39540d15d0 0x7f39540cfb10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f393c009870 tx=0x7f393c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ff69ca1afaae65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3952ffd640 1 -- 172.21.15.32:0/1191195559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0ab680 con 0x7f39540d15d0 2023-10-16T08:18:58.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3952ffd640 1 -- 172.21.15.32:0/1191195559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c0ab7e0 con 0x7f39540d15d0 2023-10-16T08:18:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3952ffd640 1 -- 172.21.15.32:0/1191195559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0abb70 con 0x7f39540d15d0 2023-10-16T08:18:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 -- 172.21.15.32:0/1191195559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39540d15d0 msgr2=0x7f39540cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 --2- 172.21.15.32:0/1191195559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39540d15d0 0x7f39540cfb10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f393c009870 tx=0x7f393c01c480 comp rx=0 tx=0).stop 2023-10-16T08:18:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 -- 172.21.15.32:0/1191195559 shutdown_connections 2023-10-16T08:18:58.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 --2- 172.21.15.32:0/1191195559 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d3010 0x7f39540c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 --2- 172.21.15.32:0/1191195559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39540d15d0 0x7f39540cfb10 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 -- 172.21.15.32:0/1191195559 >> 172.21.15.32:0/1191195559 conn(0x7f395409bc50 msgr2=0x7f39540d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 -- 172.21.15.32:0/1191195559 shutdown_connections 2023-10-16T08:18:58.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.436+0000 7f3959c32640 1 -- 172.21.15.32:0/1191195559 wait complete. 2023-10-16T08:18:58.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 Processor -- start 2023-10-16T08:18:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 -- start start 2023-10-16T08:18:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d15d0 0x7f3954157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39540d3010 0x7f39540ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 0x7f39540cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540cbac0 con 0x7f395415cb90 2023-10-16T08:18:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39541573d0 con 0x7f39540d3010 2023-10-16T08:18:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39541583b0 con 0x7f39540d15d0 2023-10-16T08:18:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 0x7f39540cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 0x7f39540cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43694/0 (socket says 172.21.15.32:43694) 2023-10-16T08:18:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 -- 172.21.15.32:0/3911443651 learned_addr learned my addr 172.21.15.32:0/3911443651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f394bfff640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39540d3010 0x7f39540ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f39537fe640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d15d0 0x7f3954157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 -- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d15d0 msgr2=0x7f3954157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d15d0 0x7f3954157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 -- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39540d3010 msgr2=0x7f39540ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39540d3010 0x7f39540ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 -- 172.21.15.32:0/3911443651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c0094d0 con 0x7f395415cb90 2023-10-16T08:18:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f39537fe640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d15d0 0x7f3954157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3953fff640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 0x7f39540cdbb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f394c007e40 tx=0x7f394c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:58.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c014620 con 0x7f395415cb90 2023-10-16T08:18:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00ee90 con 0x7f395415cb90 2023-10-16T08:18:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0149e0 con 0x7f395415cb90 2023-10-16T08:18:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39540cae40 con 0x7f395415cb90 2023-10-16T08:18:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.440+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39540cb400 con 0x7f395415cb90 2023-10-16T08:18:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.444+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39540cbc00 con 0x7f395415cb90 2023-10-16T08:18:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.444+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f394c014b40 con 0x7f395415cb90 2023-10-16T08:18:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.444+0000 7f3950ff9640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39340664a0 0x7f3934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.444+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f394c084910 con 0x7f395415cb90 2023-10-16T08:18:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.444+0000 7f39537fe640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39340664a0 0x7f3934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.444+0000 7f39537fe640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39340664a0 0x7f3934068960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3944007fd0 tx=0x7f3944009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.448+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39540cbc00 con 0x7f395415cb90 2023-10-16T08:18:58.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.608+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39540cbc00 con 0x7f395415cb90 2023-10-16T08:18:58.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.608+0000 7f3950ff9640 1 -- 172.21.15.32:0/3911443651 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f39540cbc00 con 0x7f395415cb90 2023-10-16T08:18:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:18:58.612 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:18:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39340664a0 msgr2=0x7f3934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39340664a0 0x7f3934068960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3944007fd0 tx=0x7f3944009040 comp rx=0 tx=0).stop 2023-10-16T08:18:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 msgr2=0x7f39540cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 0x7f39540cdbb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f394c007e40 tx=0x7f394c00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 shutdown_connections 2023-10-16T08:18:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f395415cb90 0x7f39540cdbb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39540d3010 0x7f39540ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39340664a0 0x7f3934068960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 --2- 172.21.15.32:0/3911443651 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39540d15d0 0x7f3954157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 >> 172.21.15.32:0/3911443651 conn(0x7f395409bc50 msgr2=0x7f39540d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 shutdown_connections 2023-10-16T08:18:58.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.612+0000 7f3959c32640 1 -- 172.21.15.32:0/3911443651 wait complete. 2023-10-16T08:18:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:18:48.142496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:18:48.637534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:18:58.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:18:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.776+0000 7f5afb59d640 1 Processor -- start 2023-10-16T08:18:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5afb59d640 1 -- start start 2023-10-16T08:18:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5afb59d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 0x7f5af40c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5afb59d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5af40c26b0 0x7f5af40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5afb59d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af415ae30 con 0x7f5af4069e90 2023-10-16T08:18:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5afb59d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40cb090 con 0x7f5af40c26b0 2023-10-16T08:18:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5afb59d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af406d470 con 0x7f5af406d010 2023-10-16T08:18:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af8b38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 0x7f5af40c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9b3a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f5af40c26b0 0x7f5af40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9339640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f5af4069e90 0x7f5af406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37496/0 (socket says 172.21.15.32:37496) 2023-10-16T08:18:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af8b38640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 0x7f5af40c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:43156/0 (socket says 172.21.15.32:43156) 2023-10-16T08:18:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9339640 1 -- 172.21.15.32:0/734138642 learned_addr learned my addr 172.21.15.32:0/734138642 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:18:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9b3a640 1 -- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 msgr2=0x7f5af40c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9b3a640 1 --2- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 0x7f5af40c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9b3a640 1 -- 172.21.15.32:0/734138642 >> v1:172.21.15.186:6789/0 conn(0x7f5af4069e90 legacy=0x7f5af406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af9b3a640 1 -- 172.21.15.32:0/734138642 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40ca510 con 0x7f5af40c26b0 2023-10-16T08:18:58.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.780+0000 7f5af8b38640 1 --2- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 0x7f5af40c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:18:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5af9b3a640 1 --2- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af40c26b0 0x7f5af40c2b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009d90 tx=0x7f5aec01f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd138998671242d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5aebfff640 1 -- 172.21.15.32:0/734138642 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0af600 con 0x7f5af40c26b0 2023-10-16T08:18:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5aebfff640 1 -- 172.21.15.32:0/734138642 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec0ab050 con 0x7f5af40c26b0 2023-10-16T08:18:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5aebfff640 1 -- 172.21.15.32:0/734138642 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0af980 con 0x7f5af40c26b0 2023-10-16T08:18:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 -- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af40c26b0 msgr2=0x7f5af40c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 --2- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af40c26b0 0x7f5af40c2b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009d90 tx=0x7f5aec01f130 comp rx=0 tx=0).stop 2023-10-16T08:18:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 -- 172.21.15.32:0/734138642 shutdown_connections 2023-10-16T08:18:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 --2- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af406d010 0x7f5af40c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 --2- 172.21.15.32:0/734138642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af40c26b0 0x7f5af40c2b40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 -- 172.21.15.32:0/734138642 >> 172.21.15.32:0/734138642 conn(0x7f5af40b0da0 msgr2=0x7f5af40ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 -- 172.21.15.32:0/734138642 shutdown_connections 2023-10-16T08:18:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 -- 172.21.15.32:0/734138642 wait complete. 2023-10-16T08:18:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.784+0000 7f5afb59d640 1 Processor -- start 2023-10-16T08:18:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- start start 2023-10-16T08:18:58.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 0x7f5af4158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af406d010 0x7f5af41f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af40c26b0 0x7f5af40c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af415c5a0 con 0x7f5af4069e90 2023-10-16T08:18:58.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af415c840 con 0x7f5af406d010 2023-10-16T08:18:58.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41f60c0 con 0x7f5af40c26b0 2023-10-16T08:18:58.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 0x7f5af4158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 0x7f5af4158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43704/0 (socket says 172.21.15.32:43704) 2023-10-16T08:18:58.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 -- 172.21.15.32:0/3319493556 learned_addr learned my addr 172.21.15.32:0/3319493556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:58.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af8b38640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af406d010 0x7f5af41f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9b3a640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af40c26b0 0x7f5af40c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 -- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af40c26b0 msgr2=0x7f5af40c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af40c26b0 0x7f5af40c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 -- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af406d010 msgr2=0x7f5af41f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af406d010 0x7f5af41f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 -- 172.21.15.32:0/3319493556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009420 con 0x7f5af4069e90 2023-10-16T08:18:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5af9339640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 0x7f5af4158d40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c500 tx=0x7f5af000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0005cf0 con 0x7f5af4069e90 2023-10-16T08:18:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0005e50 con 0x7f5af4069e90 2023-10-16T08:18:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec009a70 con 0x7f5af4069e90 2023-10-16T08:18:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00072f0 con 0x7f5af4069e90 2023-10-16T08:18:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41f6360 con 0x7f5af4069e90 2023-10-16T08:18:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.788+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af40ca7d0 con 0x7f5af4069e90 2023-10-16T08:18:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.792+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5af0007450 con 0x7f5af4069e90 2023-10-16T08:18:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.792+0000 7f5ae9ffb640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5ad80664a0 0x7f5ad8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.792+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f5af00834d0 con 0x7f5af4069e90 2023-10-16T08:18:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.792+0000 7f5af8b38640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5ad80664a0 0x7f5ad8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.792+0000 7f5af8b38640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5ad80664a0 0x7f5ad8068960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5af40c5c30 tx=0x7f5ae4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.796+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5af40ca7d0 con 0x7f5af4069e90 2023-10-16T08:18:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.952+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5af40ca7d0 con 0x7f5af4069e90 2023-10-16T08:18:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.952+0000 7f5ae9ffb640 1 -- 172.21.15.32:0/3319493556 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5af0017020 con 0x7f5af4069e90 2023-10-16T08:18:58.956 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:18:58.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5ad80664a0 msgr2=0x7f5ad8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5ad80664a0 0x7f5ad8068960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5af40c5c30 tx=0x7f5ae4009190 comp rx=0 tx=0).stop 2023-10-16T08:18:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 msgr2=0x7f5af4158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 0x7f5af4158d40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c500 tx=0x7f5af000c9c0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 shutdown_connections 2023-10-16T08:18:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5ad80664a0 0x7f5ad8068960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5af40c26b0 0x7f5af40c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5af406d010 0x7f5af41f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 --2- 172.21.15.32:0/3319493556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af4069e90 0x7f5af4158d40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 >> 172.21.15.32:0/3319493556 conn(0x7f5af40b0da0 msgr2=0x7f5af4159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 shutdown_connections 2023-10-16T08:18:58.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:58.956+0000 7f5afb59d640 1 -- 172.21.15.32:0/3319493556 wait complete. 2023-10-16T08:18:58.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:18:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 Processor -- start 2023-10-16T08:18:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 -- start start 2023-10-16T08:18:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86f80b3100 0x7f86f80b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f86f80c9020 0x7f86f80c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f806d500 con 0x7f86f80ca980 2023-10-16T08:18:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f8159f20 con 0x7f86f80b3100 2023-10-16T08:18:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86fddd0640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f806c620 con 0x7f86f80c9020 2023-10-16T08:18:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86f6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86f80b3100 0x7f86f80b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86f6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86f80b3100 0x7f86f80b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43710/0 (socket says 172.21.15.32:43710) 2023-10-16T08:18:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.120+0000 7f86f6ffd640 1 -- 172.21.15.32:0/2503734430 learned_addr learned my addr 172.21.15.32:0/2503734430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:59.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f7fff640 1 --2- 172.21.15.32:0/2503734430 >> v2:172.21.15.186:3300/0 conn(0x7f86f80c9020 0x7f86f80c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f6ffd640 1 -- 172.21.15.32:0/2503734430 >> v2:172.21.15.186:3300/0 conn(0x7f86f80c9020 msgr2=0x7f86f80c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f6ffd640 1 --2- 172.21.15.32:0/2503734430 >> v2:172.21.15.186:3300/0 conn(0x7f86f80c9020 0x7f86f80c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f6ffd640 1 -- 172.21.15.32:0/2503734430 >> v1:172.21.15.32:6789/0 conn(0x7f86f80ca980 legacy=0x7f86f80b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f6ffd640 1 -- 172.21.15.32:0/2503734430 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f806c8f0 con 0x7f86f80b3100 2023-10-16T08:18:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f67fc640 1 -- 172.21.15.32:0/2503734430 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436313381 0 0) 0x7f86f806d500 con 0x7f86f80ca980 2023-10-16T08:18:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f7fff640 1 --2- 172.21.15.32:0/2503734430 >> v2:172.21.15.186:3300/0 conn(0x7f86f80c9020 0x7f86f80c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:18:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f6ffd640 1 --2- 172.21.15.32:0/2503734430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80b3100 0x7f86f80b1dc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f86f8072670 tx=0x7f86ec01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6ace170858bad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f67fc640 1 -- 172.21.15.32:0/2503734430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0a7ca0 con 0x7f86f80b3100 2023-10-16T08:18:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f67fc640 1 -- 172.21.15.32:0/2503734430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec0a7e00 con 0x7f86f80b3100 2023-10-16T08:18:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86f67fc640 1 -- 172.21.15.32:0/2503734430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0af660 con 0x7f86f80b3100 2023-10-16T08:18:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86fddd0640 1 -- 172.21.15.32:0/2503734430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80b3100 msgr2=0x7f86f80b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.124+0000 7f86fddd0640 1 --2- 172.21.15.32:0/2503734430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80b3100 0x7f86f80b1dc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f86f8072670 tx=0x7f86ec01ebd0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- 172.21.15.32:0/2503734430 shutdown_connections 2023-10-16T08:18:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 --2- 172.21.15.32:0/2503734430 >> v2:172.21.15.186:3300/0 conn(0x7f86f80c9020 0x7f86f80c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 --2- 172.21.15.32:0/2503734430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80b3100 0x7f86f80b1dc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- 172.21.15.32:0/2503734430 >> 172.21.15.32:0/2503734430 conn(0x7f86f806a490 msgr2=0x7f86f80ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- 172.21.15.32:0/2503734430 shutdown_connections 2023-10-16T08:18:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- 172.21.15.32:0/2503734430 wait complete. 2023-10-16T08:18:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 Processor -- start 2023-10-16T08:18:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- start start 2023-10-16T08:18:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86f80b3100 0x7f86f815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 0x7f86f8071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f80ca980 0x7f86f80717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8158500 con 0x7f86f80c9020 2023-10-16T08:18:59.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f815c2b0 con 0x7f86f80ca980 2023-10-16T08:18:59.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86fddd0640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f815b6a0 con 0x7f86f80b3100 2023-10-16T08:18:59.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 0x7f86f8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 0x7f86f8071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43716/0 (socket says 172.21.15.32:43716) 2023-10-16T08:18:59.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 -- 172.21.15.32:0/771567945 learned_addr learned my addr 172.21.15.32:0/771567945 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:59.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f7fff640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f80ca980 0x7f86f80717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f77fe640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86f80b3100 0x7f86f815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 -- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86f80b3100 msgr2=0x7f86f815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86f80b3100 0x7f86f815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 -- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f80ca980 msgr2=0x7f86f80717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f80ca980 0x7f86f80717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f6ffd640 1 -- 172.21.15.32:0/771567945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec00b770 con 0x7f86f80c9020 2023-10-16T08:18:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.128+0000 7f86f7fff640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f80ca980 0x7f86f80717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86f6ffd640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 0x7f86f8071280 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f86ec01b7e0 tx=0x7f86ec01b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0ba660 con 0x7f86f80c9020 2023-10-16T08:18:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec0ba7e0 con 0x7f86f80c9020 2023-10-16T08:18:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0bab90 con 0x7f86f80c9020 2023-10-16T08:18:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f815b980 con 0x7f86f80c9020 2023-10-16T08:18:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f815bf40 con 0x7f86f80c9020 2023-10-16T08:18:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f8073fa0 con 0x7f86f80c9020 2023-10-16T08:18:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86ec0c2360 con 0x7f86f80c9020 2023-10-16T08:18:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86dffff640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86e40664a0 0x7f86e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f86ec11ee60 con 0x7f86f80c9020 2023-10-16T08:18:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.132+0000 7f86f77fe640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86e40664a0 0x7f86e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.136+0000 7f86f77fe640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86e40664a0 0x7f86e4068960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f86e8001670 tx=0x7f86e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.136+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86ec0ec290 con 0x7f86f80c9020 2023-10-16T08:18:59.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.292+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86f815c420 con 0x7f86f80c9020 2023-10-16T08:18:59.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86dffff640 1 -- 172.21.15.32:0/771567945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v54) v1 ==== 109+0+11 (secure 0 0 0) 0x7f86ec0bf0b0 con 0x7f86f80c9020 2023-10-16T08:18:59.297 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:18:59.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86e40664a0 msgr2=0x7f86e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86e40664a0 0x7f86e4068960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f86e8001670 tx=0x7f86e8009040 comp rx=0 tx=0).stop 2023-10-16T08:18:59.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 msgr2=0x7f86f8071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 0x7f86f8071280 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f86ec01b7e0 tx=0x7f86ec01b970 comp rx=0 tx=0).stop 2023-10-16T08:18:59.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 shutdown_connections 2023-10-16T08:18:59.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f80ca980 0x7f86f80717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86f80c9020 0x7f86f8071280 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f86e40664a0 0x7f86e4068960 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 --2- 172.21.15.32:0/771567945 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f86f80b3100 0x7f86f815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 >> 172.21.15.32:0/771567945 conn(0x7f86f806a490 msgr2=0x7f86f80ab0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:59.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 shutdown_connections 2023-10-16T08:18:59.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.296+0000 7f86fddd0640 1 -- 172.21.15.32:0/771567945 wait complete. 2023-10-16T08:18:59.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:18:59.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 Processor -- start 2023-10-16T08:18:59.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 -- start start 2023-10-16T08:18:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f176c0b84e0 0x7f176c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0d1670 con 0x7f176c0d0040 2023-10-16T08:18:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0d1460 con 0x7f176c0b84e0 2023-10-16T08:18:59.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1773617640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0d04b0 con 0x7f176c15cba0 2023-10-16T08:18:59.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1770bb2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f176c0b84e0 0x7f176c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1770bb2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f176c0b84e0 0x7f176c0cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43726/0 (socket says 172.21.15.32:43726) 2023-10-16T08:18:59.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f1770bb2640 1 -- 172.21.15.32:0/256758295 learned_addr learned my addr 172.21.15.32:0/256758295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:59.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.464+0000 7f17713b3640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1770bb2640 1 -- 172.21.15.32:0/256758295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 msgr2=0x7f176c0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:18:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1770bb2640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1770bb2640 1 -- 172.21.15.32:0/256758295 >> v1:172.21.15.32:6789/0 conn(0x7f176c0d0040 legacy=0x7f176c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1770bb2640 1 -- 172.21.15.32:0/256758295 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0d1950 con 0x7f176c0b84e0 2023-10-16T08:18:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f17713b3640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:18:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1763fff640 1 -- 172.21.15.32:0/256758295 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102625797 0 0) 0x7f176c0d1670 con 0x7f176c0d0040 2023-10-16T08:18:59.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1770bb2640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c0cfb00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f175c009870 tx=0x7f175c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca65222641ccb1b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1763fff640 1 -- 172.21.15.32:0/256758295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0ab660 con 0x7f176c0b84e0 2023-10-16T08:18:59.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1763fff640 1 -- 172.21.15.32:0/256758295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c0ab7c0 con 0x7f176c0b84e0 2023-10-16T08:18:59.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1763fff640 1 -- 172.21.15.32:0/256758295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0abb50 con 0x7f176c0b84e0 2023-10-16T08:18:59.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1773617640 1 -- 172.21.15.32:0/256758295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 msgr2=0x7f176c0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1773617640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c0cfb00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f175c009870 tx=0x7f175c01c540 comp rx=0 tx=0).stop 2023-10-16T08:18:59.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1773617640 1 -- 172.21.15.32:0/256758295 shutdown_connections 2023-10-16T08:18:59.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1773617640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c0cfb00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1773617640 1 --2- 172.21.15.32:0/256758295 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.468+0000 7f1773617640 1 -- 172.21.15.32:0/256758295 >> 172.21.15.32:0/256758295 conn(0x7f176c0719c0 msgr2=0x7f176c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:59.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- 172.21.15.32:0/256758295 shutdown_connections 2023-10-16T08:18:59.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- 172.21.15.32:0/256758295 wait complete. 2023-10-16T08:18:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 Processor -- start 2023-10-16T08:18:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- start start 2023-10-16T08:18:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0d0040 0x7f176c0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c1480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cbfc0 con 0x7f176c0b84e0 2023-10-16T08:18:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cca50 con 0x7f176c0d0040 2023-10-16T08:18:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cbc50 con 0x7f176c15cba0 2023-10-16T08:18:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43728/0 (socket says 172.21.15.32:43728) 2023-10-16T08:18:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 -- 172.21.15.32:0/4051138056 learned_addr learned my addr 172.21.15.32:0/4051138056 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 -- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 msgr2=0x7f176c1480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c1480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 -- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0d0040 msgr2=0x7f176c0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0d0040 0x7f176c0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 -- 172.21.15.32:0/4051138056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0094d0 con 0x7f176c0b84e0 2023-10-16T08:18:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f17713b3640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c149ec0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f176800d800 tx=0x7f176800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17680096c0 con 0x7f176c0b84e0 2023-10-16T08:18:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768009820 con 0x7f176c0b84e0 2023-10-16T08:18:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768009bd0 con 0x7f176c0b84e0 2023-10-16T08:18:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.472+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c148600 con 0x7f176c0b84e0 2023-10-16T08:18:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c14ecd0 con 0x7f176c0b84e0 2023-10-16T08:18:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c0672e0 con 0x7f176c0b84e0 2023-10-16T08:18:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1768009d30 con 0x7f176c0b84e0 2023-10-16T08:18:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1761ffb640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17500664a0 0x7f1750068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f1768084780 con 0x7f176c0b84e0 2023-10-16T08:18:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1770bb2640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17500664a0 0x7f1750068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.476+0000 7f1770bb2640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17500664a0 0x7f1750068960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f175c0af000 tx=0x7f175c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.480+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1768051a00 con 0x7f176c0b84e0 2023-10-16T08:18:59.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f176c0cbdc0 con 0x7f176c0b84e0 2023-10-16T08:18:59.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1761ffb640 1 -- 172.21.15.32:0/4051138056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v54) v1 ==== 105+0+11 (secure 0 0 0) 0x7f1768055020 con 0x7f176c0b84e0 2023-10-16T08:18:59.642 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:18:59.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17500664a0 msgr2=0x7f1750068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17500664a0 0x7f1750068960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f175c0af000 tx=0x7f175c008040 comp rx=0 tx=0).stop 2023-10-16T08:18:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 msgr2=0x7f176c149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c149ec0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f176800d800 tx=0x7f176800dcc0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 shutdown_connections 2023-10-16T08:18:59.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f17500664a0 0x7f1750068960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f176c15cba0 0x7f176c1480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0d0040 0x7f176c0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 --2- 172.21.15.32:0/4051138056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c0b84e0 0x7f176c149ec0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 >> 172.21.15.32:0/4051138056 conn(0x7f176c0719c0 msgr2=0x7f176c0b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:59.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 shutdown_connections 2023-10-16T08:18:59.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.640+0000 7f1773617640 1 -- 172.21.15.32:0/4051138056 wait complete. 2023-10-16T08:18:59.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:18:59.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd51751d640 1 Processor -- start 2023-10-16T08:18:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd51751d640 1 -- start start 2023-10-16T08:18:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd51751d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5100ae960 0x7fd5100ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd51751d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100cdf30 con 0x7fd5100cf0c0 2023-10-16T08:18:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd51751d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100cd320 con 0x7fd5100b4fb0 2023-10-16T08:18:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd51751d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd510071740 con 0x7fd5100ae960 2023-10-16T08:18:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd5152b9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5100ae960 0x7fd5100ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd5152b9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5100ae960 0x7fd5100ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43742/0 (socket says 172.21.15.32:43742) 2023-10-16T08:18:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.804+0000 7fd5152b9640 1 -- 172.21.15.32:0/1151771816 learned_addr learned my addr 172.21.15.32:0/1151771816 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd5152b9640 1 -- 172.21.15.32:0/1151771816 >> v1:172.21.15.186:6789/0 conn(0x7fd5100b4fb0 legacy=0x7fd5100ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd5152b9640 1 -- 172.21.15.32:0/1151771816 >> v1:172.21.15.32:6789/0 conn(0x7fd5100cf0c0 legacy=0x7fd510159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd5152b9640 1 -- 172.21.15.32:0/1151771816 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100cd600 con 0x7fd5100ae960 2023-10-16T08:18:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd507fff640 1 -- 172.21.15.32:0/1151771816 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554863533 0 0) 0x7fd5100cdf30 con 0x7fd5100cf0c0 2023-10-16T08:18:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd5152b9640 1 --2- 172.21.15.32:0/1151771816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100ae960 0x7fd5100ce3c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0098a0 tx=0x7fd50c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50be0715948f6622 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd507fff640 1 -- 172.21.15.32:0/1151771816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c004230 con 0x7fd5100ae960 2023-10-16T08:18:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd507fff640 1 -- 172.21.15.32:0/1151771816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c0a7050 con 0x7fd5100ae960 2023-10-16T08:18:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd507fff640 1 -- 172.21.15.32:0/1151771816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c003620 con 0x7fd5100ae960 2023-10-16T08:18:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 -- 172.21.15.32:0/1151771816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100ae960 msgr2=0x7fd5100ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 --2- 172.21.15.32:0/1151771816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100ae960 0x7fd5100ce3c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0098a0 tx=0x7fd50c01c520 comp rx=0 tx=0).stop 2023-10-16T08:18:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 -- 172.21.15.32:0/1151771816 shutdown_connections 2023-10-16T08:18:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 --2- 172.21.15.32:0/1151771816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100ae960 0x7fd5100ce3c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 -- 172.21.15.32:0/1151771816 >> 172.21.15.32:0/1151771816 conn(0x7fd51006d310 msgr2=0x7fd51006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 -- 172.21.15.32:0/1151771816 shutdown_connections 2023-10-16T08:18:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.808+0000 7fd51751d640 1 -- 172.21.15.32:0/1151771816 wait complete. 2023-10-16T08:18:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 Processor -- start 2023-10-16T08:18:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 -- start start 2023-10-16T08:18:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5100ae960 0x7fd51014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 0x7fd51014c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5100cf0c0 0x7fd5100cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100afae0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100b1750 con 0x7fd5100cf0c0 2023-10-16T08:18:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100b02f0 con 0x7fd5100ae960 2023-10-16T08:18:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 0x7fd51014c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 0x7fd51014c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43754/0 (socket says 172.21.15.32:43754) 2023-10-16T08:18:59.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 -- 172.21.15.32:0/2938258443 learned_addr learned my addr 172.21.15.32:0/2938258443 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:18:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd5152b9640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5100ae960 0x7fd51014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd515aba640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5100cf0c0 0x7fd5100cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 -- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5100ae960 msgr2=0x7fd51014c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5100ae960 0x7fd51014c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 -- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5100cf0c0 msgr2=0x7fd5100cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5100cf0c0 0x7fd5100cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 -- 172.21.15.32:0/2938258443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0094d0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd514ab8640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 0x7fd51014c620 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c3f0 tx=0x7fd50000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000052d0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd500005430 con 0x7fd5100b4fb0 2023-10-16T08:18:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500003840 con 0x7fd5100b4fb0 2023-10-16T08:18:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.812+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5100afd30 con 0x7fd5100b4fb0 2023-10-16T08:18:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5100b19d0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd510074010 con 0x7fd5100b4fb0 2023-10-16T08:18:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd5000039a0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd505ffb640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4f40664a0 0x7fd4f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:18:59.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fd5000834b0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd5152b9640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4f40664a0 0x7fd4f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:18:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.816+0000 7fd5152b9640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4f40664a0 0x7fd4f4068960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0ad000 tx=0x7fd50c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:18:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.820+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd500050870 con 0x7fd5100b4fb0 2023-10-16T08:18:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.992+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5100af8c0 con 0x7fd5100b4fb0 2023-10-16T08:18:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.992+0000 7fd505ffb640 1 -- 172.21.15.32:0/2938258443 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd500053e90 con 0x7fd5100b4fb0 2023-10-16T08:18:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:18:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4f40664a0 msgr2=0x7fd4f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4f40664a0 0x7fd4f4068960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0ad000 tx=0x7fd50c0a6000 comp rx=0 tx=0).stop 2023-10-16T08:18:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 msgr2=0x7fd51014c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:18:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 0x7fd51014c620 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd50000c3f0 tx=0x7fd50000c8b0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 shutdown_connections 2023-10-16T08:18:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5100cf0c0 0x7fd5100cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5100b4fb0 0x7fd51014c620 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd4f40664a0 0x7fd4f4068960 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 --2- 172.21.15.32:0/2938258443 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd5100ae960 0x7fd51014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:18:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 >> 172.21.15.32:0/2938258443 conn(0x7fd51006d310 msgr2=0x7fd5100ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:18:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 shutdown_connections 2023-10-16T08:19:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:18:59.996+0000 7fd51751d640 1 -- 172.21.15.32:0/2938258443 wait complete. 2023-10-16T08:19:00.019 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:00.019 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:00.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 Processor -- start 2023-10-16T08:19:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 -- start start 2023-10-16T08:19:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f530815cb90 0x7f53080cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53080d3010 0x7f53080d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53080ce770 con 0x7f53080d1400 2023-10-16T08:19:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53080d1890 con 0x7f530815cb90 2023-10-16T08:19:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f5310646640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080d0600 con 0x7f53080d3010 2023-10-16T08:19:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530dbe1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53080d3010 0x7f53080d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530ebe3640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f53080d1400 0x7f53080cd3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37514/0 (socket says 172.21.15.32:37514) 2023-10-16T08:19:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530ebe3640 1 -- 172.21.15.32:0/2078735837 learned_addr learned my addr 172.21.15.32:0/2078735837 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530e3e2640 1 --2- 172.21.15.32:0/2078735837 >> v2:172.21.15.186:3300/0 conn(0x7f530815cb90 0x7f53080cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530d3e0640 1 -- 172.21.15.32:0/2078735837 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662804132 0 0) 0x7f53080ce770 con 0x7f53080d1400 2023-10-16T08:19:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530d3e0640 1 -- 172.21.15.32:0/2078735837 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f4003530 con 0x7f53080d1400 2023-10-16T08:19:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530e3e2640 1 -- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53080d3010 msgr2=0x7f53080d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530e3e2640 1 --2- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53080d3010 0x7f53080d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530e3e2640 1 -- 172.21.15.32:0/2078735837 >> v1:172.21.15.186:6789/0 conn(0x7f53080d1400 legacy=0x7f53080cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530e3e2640 1 -- 172.21.15.32:0/2078735837 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53080c6450 con 0x7f530815cb90 2023-10-16T08:19:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.216+0000 7f530dbe1640 1 --2- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53080d3010 0x7f53080d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f530e3e2640 1 --2- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530815cb90 0x7f53080cfb10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53040097a0 tx=0x7f530401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1023d3b2ffb01cf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f530d3e0640 1 -- 172.21.15.32:0/2078735837 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040ac600 con 0x7f530815cb90 2023-10-16T08:19:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f530d3e0640 1 -- 172.21.15.32:0/2078735837 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53040a8050 con 0x7f530815cb90 2023-10-16T08:19:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f530d3e0640 1 -- 172.21.15.32:0/2078735837 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040ac980 con 0x7f530815cb90 2023-10-16T08:19:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 -- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530815cb90 msgr2=0x7f53080cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 --2- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530815cb90 0x7f53080cfb10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53040097a0 tx=0x7f530401c3a0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 -- 172.21.15.32:0/2078735837 shutdown_connections 2023-10-16T08:19:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 --2- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f53080d3010 0x7f53080d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 --2- 172.21.15.32:0/2078735837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f530815cb90 0x7f53080cfb10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 -- 172.21.15.32:0/2078735837 >> 172.21.15.32:0/2078735837 conn(0x7f530809bc50 msgr2=0x7f53080d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 -- 172.21.15.32:0/2078735837 shutdown_connections 2023-10-16T08:19:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.220+0000 7f5310646640 1 -- 172.21.15.32:0/2078735837 wait complete. 2023-10-16T08:19:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 Processor -- start 2023-10-16T08:19:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 -- start start 2023-10-16T08:19:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53080d1400 0x7f530814f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 0x7f530814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530815cb90 0x7f53080c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530815ac20 con 0x7f53080d3010 2023-10-16T08:19:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530814b7a0 con 0x7f53080d1400 2023-10-16T08:19:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5308157c00 con 0x7f530815cb90 2023-10-16T08:19:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530dbe1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 0x7f530814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53080d1400 0x7f530814f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530dbe1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 0x7f530814b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43762/0 (socket says 172.21.15.32:43762) 2023-10-16T08:19:00.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530dbe1640 1 -- 172.21.15.32:0/780688549 learned_addr learned my addr 172.21.15.32:0/780688549 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:00.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530ebe3640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530815cb90 0x7f53080c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 -- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530815cb90 msgr2=0x7f53080c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530815cb90 0x7f53080c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 -- 172.21.15.32:0/780688549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 msgr2=0x7f530814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 0x7f530814b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 -- 172.21.15.32:0/780688549 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f8009420 con 0x7f53080d1400 2023-10-16T08:19:00.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530dbe1640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 0x7f530814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f530e3e2640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53080d1400 0x7f530814f4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53040ac000 tx=0x7f530401abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040059a0 con 0x7f53080d1400 2023-10-16T08:19:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5304005b00 con 0x7f53080d1400 2023-10-16T08:19:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53040093b0 con 0x7f53080d1400 2023-10-16T08:19:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.224+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304006070 con 0x7f53080d1400 2023-10-16T08:19:00.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53081581c0 con 0x7f53080d1400 2023-10-16T08:19:00.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5308074010 con 0x7f53080d1400 2023-10-16T08:19:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53040b3040 con 0x7f53080d1400 2023-10-16T08:19:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f52feffd640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f52ec0664d0 0x7f52ec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f530411e710 con 0x7f53080d1400 2023-10-16T08:19:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f530dbe1640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f52ec0664d0 0x7f52ec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.228+0000 7f530dbe1640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f52ec0664d0 0x7f52ec068990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f53080c8ea0 tx=0x7f52f8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.232+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f53040ebb50 con 0x7f53080d1400 2023-10-16T08:19:00.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.392+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f530809c030 con 0x7f53080d1400 2023-10-16T08:19:00.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.392+0000 7f52feffd640 1 -- 172.21.15.32:0/780688549 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f53040ef170 con 0x7f53080d1400 2023-10-16T08:19:00.396 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:00.397 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:00.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f52ec0664d0 msgr2=0x7f52ec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f52ec0664d0 0x7f52ec068990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f53080c8ea0 tx=0x7f52f8009190 comp rx=0 tx=0).stop 2023-10-16T08:19:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53080d1400 msgr2=0x7f530814f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53080d1400 0x7f530814f4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53040ac000 tx=0x7f530401abe0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 shutdown_connections 2023-10-16T08:19:00.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f52ec0664d0 0x7f52ec068990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f530815cb90 0x7f53080c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080d3010 0x7f530814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 --2- 172.21.15.32:0/780688549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53080d1400 0x7f530814f4b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 >> 172.21.15.32:0/780688549 conn(0x7f530809bc50 msgr2=0x7f53081556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 shutdown_connections 2023-10-16T08:19:00.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.396+0000 7f5310646640 1 -- 172.21.15.32:0/780688549 wait complete. 2023-10-16T08:19:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:18:48.142496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:18:48.637534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:19:00.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:19:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.560+0000 7f962d332640 1 Processor -- start 2023-10-16T08:19:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.560+0000 7f962d332640 1 -- start start 2023-10-16T08:19:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962d332640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f96280cf210 0x7f96280cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962d332640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96280d3320 0x7f96280ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962d332640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96280cfb30 con 0x7f962815cdc0 2023-10-16T08:19:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962d332640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280cea10 con 0x7f96280d3320 2023-10-16T08:19:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962d332640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280d0880 con 0x7f96280cf210 2023-10-16T08:19:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f96277fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f962815cdc0 0x7f96280b2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42146/0 (socket says 172.21.15.32:42146) 2023-10-16T08:19:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f96277fe640 1 -- 172.21.15.32:0/1178968222 learned_addr learned my addr 172.21.15.32:0/1178968222 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962cb31640 1 --2- 172.21.15.32:0/1178968222 >> v2:172.21.15.32:3300/0 conn(0x7f96280d3320 0x7f96280ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f9627fff640 1 --2- 172.21.15.32:0/1178968222 >> v2:172.21.15.186:3300/0 conn(0x7f96280cf210 0x7f96280cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f9626ffd640 1 -- 172.21.15.32:0/1178968222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632542176 0 0) 0x7f96280cfb30 con 0x7f962815cdc0 2023-10-16T08:19:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f9626ffd640 1 -- 172.21.15.32:0/1178968222 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9610003530 con 0x7f962815cdc0 2023-10-16T08:19:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962cb31640 1 -- 172.21.15.32:0/1178968222 >> v2:172.21.15.186:3300/0 conn(0x7f96280cf210 msgr2=0x7f96280cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962cb31640 1 --2- 172.21.15.32:0/1178968222 >> v2:172.21.15.186:3300/0 conn(0x7f96280cf210 0x7f96280cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962cb31640 1 -- 172.21.15.32:0/1178968222 >> v1:172.21.15.32:6789/0 conn(0x7f962815cdc0 legacy=0x7f96280b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.564+0000 7f962cb31640 1 -- 172.21.15.32:0/1178968222 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280b1c00 con 0x7f96280d3320 2023-10-16T08:19:00.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962cb31640 1 --2- 172.21.15.32:0/1178968222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f96280ce4d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f962808b880 tx=0x7f962001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b137b9dcbe3f319 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f9626ffd640 1 -- 172.21.15.32:0/1178968222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96200a5910 con 0x7f96280d3320 2023-10-16T08:19:00.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f9626ffd640 1 -- 172.21.15.32:0/1178968222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96200a5a70 con 0x7f96280d3320 2023-10-16T08:19:00.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f9626ffd640 1 -- 172.21.15.32:0/1178968222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96200a5e00 con 0x7f96280d3320 2023-10-16T08:19:00.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 -- 172.21.15.32:0/1178968222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 msgr2=0x7f96280ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 --2- 172.21.15.32:0/1178968222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f96280ce4d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f962808b880 tx=0x7f962001c850 comp rx=0 tx=0).stop 2023-10-16T08:19:00.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 -- 172.21.15.32:0/1178968222 shutdown_connections 2023-10-16T08:19:00.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 --2- 172.21.15.32:0/1178968222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f96280ce4d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 --2- 172.21.15.32:0/1178968222 >> v2:172.21.15.186:3300/0 conn(0x7f96280cf210 0x7f96280cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 -- 172.21.15.32:0/1178968222 >> 172.21.15.32:0/1178968222 conn(0x7f96280d2800 msgr2=0x7f96280d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:00.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 -- 172.21.15.32:0/1178968222 shutdown_connections 2023-10-16T08:19:00.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 -- 172.21.15.32:0/1178968222 wait complete. 2023-10-16T08:19:00.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 Processor -- start 2023-10-16T08:19:00.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.568+0000 7f962d332640 1 -- start start 2023-10-16T08:19:00.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96280cf210 0x7f962815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f9628157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f962815cdc0 0x7f96280c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962815c650 con 0x7f96280d3320 2023-10-16T08:19:00.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962815ddb0 con 0x7f962815cdc0 2023-10-16T08:19:00.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962815d1a0 con 0x7f96280cf210 2023-10-16T08:19:00.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f9628157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f9628157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43772/0 (socket says 172.21.15.32:43772) 2023-10-16T08:19:00.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 -- 172.21.15.32:0/1949776928 learned_addr learned my addr 172.21.15.32:0/1949776928 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:00.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f9627fff640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96280cf210 0x7f962815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962cb31640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f962815cdc0 0x7f96280c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 -- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96280cf210 msgr2=0x7f962815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96280cf210 0x7f962815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 -- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f962815cdc0 msgr2=0x7f96280c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f962815cdc0 0x7f96280c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 -- 172.21.15.32:0/1949776928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9620009a70 con 0x7f96280d3320 2023-10-16T08:19:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f96277fe640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f9628157830 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f961c00c420 tx=0x7f961c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c005cf0 con 0x7f96280d3320 2023-10-16T08:19:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c005e50 con 0x7f96280d3320 2023-10-16T08:19:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c0072d0 con 0x7f96280d3320 2023-10-16T08:19:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962815d480 con 0x7f96280d3320 2023-10-16T08:19:00.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962815da40 con 0x7f96280d3320 2023-10-16T08:19:00.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962804e9c0 con 0x7f96280d3320 2023-10-16T08:19:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.572+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f961c007430 con 0x7f96280d3320 2023-10-16T08:19:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.576+0000 7f9624ff9640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96140664d0 0x7f9614068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.576+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f961c083580 con 0x7f96280d3320 2023-10-16T08:19:00.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.576+0000 7f9627fff640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96140664d0 0x7f9614068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.576+0000 7f9627fff640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96140664d0 0x7f9614068990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9618009b40 tx=0x7f9618006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.580+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f961c0508c0 con 0x7f96280d3320 2023-10-16T08:19:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.736+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96280d0bc0 con 0x7f96280d3320 2023-10-16T08:19:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.736+0000 7f9624ff9640 1 -- 172.21.15.32:0/1949776928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f961c0170a0 con 0x7f96280d3320 2023-10-16T08:19:00.741 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:19:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96140664d0 msgr2=0x7f9614068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96140664d0 0x7f9614068990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9618009b40 tx=0x7f9618006f90 comp rx=0 tx=0).stop 2023-10-16T08:19:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 msgr2=0x7f9628157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f9628157830 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f961c00c420 tx=0x7f961c00c8e0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 shutdown_connections 2023-10-16T08:19:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f962815cdc0 0x7f96280c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96280d3320 0x7f9628157830 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f96140664d0 0x7f9614068990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 --2- 172.21.15.32:0/1949776928 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f96280cf210 0x7f962815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 >> 172.21.15.32:0/1949776928 conn(0x7f96280d2800 msgr2=0x7f96280d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 shutdown_connections 2023-10-16T08:19:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.740+0000 7f962d332640 1 -- 172.21.15.32:0/1949776928 wait complete. 2023-10-16T08:19:00.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:19:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.904+0000 7f517f59e640 1 Processor -- start 2023-10-16T08:19:00.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.904+0000 7f517f59e640 1 -- start start 2023-10-16T08:19:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517f59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51800cf210 0x7f51800cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517f59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51800d3320 0x7f51800ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517f59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800cfb30 con 0x7f518015cdc0 2023-10-16T08:19:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517f59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800cea10 con 0x7f51800cf210 2023-10-16T08:19:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517f59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800d0880 con 0x7f51800d3320 2023-10-16T08:19:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51800cf210 0x7f51800cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517dd9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f518015cdc0 0x7f51800b2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42150/0 (socket says 172.21.15.32:42150) 2023-10-16T08:19:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517dd9b640 1 -- 172.21.15.32:0/3927972280 learned_addr learned my addr 172.21.15.32:0/3927972280 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517ed9d640 1 --2- 172.21.15.32:0/3927972280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51800d3320 0x7f51800ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517e59c640 1 -- 172.21.15.32:0/3927972280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51800d3320 msgr2=0x7f51800ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517e59c640 1 --2- 172.21.15.32:0/3927972280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51800d3320 0x7f51800ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517e59c640 1 -- 172.21.15.32:0/3927972280 >> v1:172.21.15.32:6789/0 conn(0x7f518015cdc0 legacy=0x7f51800b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517e59c640 1 -- 172.21.15.32:0/3927972280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800b1c00 con 0x7f51800cf210 2023-10-16T08:19:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.908+0000 7f517d59a640 1 -- 172.21.15.32:0/3927972280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485230484 0 0) 0x7f51800cfb30 con 0x7f518015cdc0 2023-10-16T08:19:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517e59c640 1 --2- 172.21.15.32:0/3927972280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800cf210 0x7f51800cf5f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f51700097a0 tx=0x7f517001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c510b0ba83231c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517d59a640 1 -- 172.21.15.32:0/3927972280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700ab600 con 0x7f51800cf210 2023-10-16T08:19:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517d59a640 1 -- 172.21.15.32:0/3927972280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51700a7050 con 0x7f51800cf210 2023-10-16T08:19:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517d59a640 1 -- 172.21.15.32:0/3927972280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700ab980 con 0x7f51800cf210 2023-10-16T08:19:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 -- 172.21.15.32:0/3927972280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800cf210 msgr2=0x7f51800cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 --2- 172.21.15.32:0/3927972280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800cf210 0x7f51800cf5f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f51700097a0 tx=0x7f517001c430 comp rx=0 tx=0).stop 2023-10-16T08:19:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 -- 172.21.15.32:0/3927972280 shutdown_connections 2023-10-16T08:19:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 --2- 172.21.15.32:0/3927972280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51800d3320 0x7f51800ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 --2- 172.21.15.32:0/3927972280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800cf210 0x7f51800cf5f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 -- 172.21.15.32:0/3927972280 >> 172.21.15.32:0/3927972280 conn(0x7f51800d2800 msgr2=0x7f51800d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 -- 172.21.15.32:0/3927972280 shutdown_connections 2023-10-16T08:19:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 -- 172.21.15.32:0/3927972280 wait complete. 2023-10-16T08:19:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 Processor -- start 2023-10-16T08:19:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.912+0000 7f517f59e640 1 -- start start 2023-10-16T08:19:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51800cf210 0x7f51800afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 0x7f51801583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518015cdc0 0x7f51800cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180158e80 con 0x7f51800d3320 2023-10-16T08:19:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180159160 con 0x7f51800cf210 2023-10-16T08:19:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5180159440 con 0x7f518015cdc0 2023-10-16T08:19:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 0x7f51801583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 0x7f51801583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43782/0 (socket says 172.21.15.32:43782) 2023-10-16T08:19:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 -- 172.21.15.32:0/1436510528 learned_addr learned my addr 172.21.15.32:0/1436510528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517ed9d640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518015cdc0 0x7f51800cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517e59c640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51800cf210 0x7f51800afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 -- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518015cdc0 msgr2=0x7f51800cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518015cdc0 0x7f51800cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 -- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51800cf210 msgr2=0x7f51800afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51800cf210 0x7f51800afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 -- 172.21.15.32:0/1436510528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51700093b0 con 0x7f51800d3320 2023-10-16T08:19:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517dd9b640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 0x7f51801583a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f517400c480 tx=0x7f517400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740074e0 con 0x7f51800d3320 2023-10-16T08:19:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174007660 con 0x7f51800d3320 2023-10-16T08:19:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174004a30 con 0x7f51800d3320 2023-10-16T08:19:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5180159720 con 0x7f51800d3320 2023-10-16T08:19:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518014d270 con 0x7f51800d3320 2023-10-16T08:19:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518014af80 con 0x7f51800d3320 2023-10-16T08:19:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.916+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5174004b90 con 0x7f51800d3320 2023-10-16T08:19:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.920+0000 7f516effd640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51600664a0 0x7f5160068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.920+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f51740830f0 con 0x7f51800d3320 2023-10-16T08:19:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.920+0000 7f517e59c640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51600664a0 0x7f5160068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.920+0000 7f517e59c640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51600664a0 0x7f5160068960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f51800b88b0 tx=0x7f51700a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:00.924+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f518014af80 con 0x7f51800d3320 2023-10-16T08:19:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f518014af80 con 0x7f51800d3320 2023-10-16T08:19:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f516effd640 1 -- 172.21.15.32:0/1436510528 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v54) v1 ==== 109+0+11 (secure 0 0 0) 0x7f518014af80 con 0x7f51800d3320 2023-10-16T08:19:01.082 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:19:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51600664a0 msgr2=0x7f5160068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51600664a0 0x7f5160068960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f51800b88b0 tx=0x7f51700a6000 comp rx=0 tx=0).stop 2023-10-16T08:19:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 msgr2=0x7f51801583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 0x7f51801583a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f517400c480 tx=0x7f517400c940 comp rx=0 tx=0).stop 2023-10-16T08:19:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 shutdown_connections 2023-10-16T08:19:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51600664a0 0x7f5160068960 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f518015cdc0 0x7f51800cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51800d3320 0x7f51801583a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 --2- 172.21.15.32:0/1436510528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51800cf210 0x7f51800afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 >> 172.21.15.32:0/1436510528 conn(0x7f51800d2800 msgr2=0x7f51800d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 shutdown_connections 2023-10-16T08:19:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.080+0000 7f517f59e640 1 -- 172.21.15.32:0/1436510528 wait complete. 2023-10-16T08:19:01.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:19:01.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38fcb1d640 1 Processor -- start 2023-10-16T08:19:01.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38fcb1d640 1 -- start start 2023-10-16T08:19:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38fcb1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f815cba0 0x7f38f80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38fcb1d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80d1670 con 0x7f38f80d0210 2023-10-16T08:19:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38fcb1d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80d1950 con 0x7f38f80b84e0 2023-10-16T08:19:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38fcb1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80d1460 con 0x7f38f815cba0 2023-10-16T08:19:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f6d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38f80d0210 0x7f38f80b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42156/0 (socket says 172.21.15.32:42156) 2023-10-16T08:19:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f6d9d640 1 -- 172.21.15.32:0/2171619505 learned_addr learned my addr 172.21.15.32:0/2171619505 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f659c640 1 --2- 172.21.15.32:0/2171619505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f815cba0 0x7f38f80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979745749 0 0) 0x7f38f80d1670 con 0x7f38f80d0210 2023-10-16T08:19:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4003530 con 0x7f38f80d0210 2023-10-16T08:19:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774565853 0 0) 0x7f38f80d1950 con 0x7f38f80b84e0 2023-10-16T08:19:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f80d1670 con 0x7f38f80b84e0 2023-10-16T08:19:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f659c640 1 -- 172.21.15.32:0/2171619505 >> v1:172.21.15.186:6789/0 conn(0x7f38f80b84e0 legacy=0x7f38f80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f659c640 1 -- 172.21.15.32:0/2171619505 >> v1:172.21.15.32:6789/0 conn(0x7f38f80d0210 legacy=0x7f38f80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.248+0000 7f38f659c640 1 -- 172.21.15.32:0/2171619505 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80d1c30 con 0x7f38f815cba0 2023-10-16T08:19:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38f659c640 1 --2- 172.21.15.32:0/2171619505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f815cba0 0x7f38f80d0f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097f0 tx=0x7f38e801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86428f38deb3b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80aa620 con 0x7f38f815cba0 2023-10-16T08:19:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80aa780 con 0x7f38f815cba0 2023-10-16T08:19:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38f5d9b640 1 -- 172.21.15.32:0/2171619505 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80aab10 con 0x7f38f815cba0 2023-10-16T08:19:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2171619505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f815cba0 msgr2=0x7f38f80d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2171619505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f815cba0 0x7f38f80d0f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097f0 tx=0x7f38e801c4c0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2171619505 shutdown_connections 2023-10-16T08:19:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2171619505 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f815cba0 0x7f38f80d0f20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2171619505 >> 172.21.15.32:0/2171619505 conn(0x7f38f80719c0 msgr2=0x7f38f8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2171619505 shutdown_connections 2023-10-16T08:19:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.252+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2171619505 wait complete. 2023-10-16T08:19:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 Processor -- start 2023-10-16T08:19:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- start start 2023-10-16T08:19:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 0x7f38f80c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f80d0210 0x7f38f81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38f815cba0 0x7f38f814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8145c10 con 0x7f38f80b84e0 2023-10-16T08:19:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8154980 con 0x7f38f815cba0 2023-10-16T08:19:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8154c60 con 0x7f38f80d0210 2023-10-16T08:19:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 0x7f38f80c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 0x7f38f80c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43786/0 (socket says 172.21.15.32:43786) 2023-10-16T08:19:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 -- 172.21.15.32:0/2223683863 learned_addr learned my addr 172.21.15.32:0/2223683863 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f6d9d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38f815cba0 0x7f38f814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38effff640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f80d0210 0x7f38f81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 -- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f80d0210 msgr2=0x7f38f81542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f80d0210 0x7f38f81542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 -- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38f815cba0 msgr2=0x7f38f814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38f815cba0 0x7f38f814d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 -- 172.21.15.32:0/2223683863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009420 con 0x7f38f80b84e0 2023-10-16T08:19:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f6d9d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38f815cba0 0x7f38f814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38f659c640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 0x7f38f80c59a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f38e80ac040 tx=0x7f38e801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80aadc0 con 0x7f38f80b84e0 2023-10-16T08:19:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80be410 con 0x7f38f80b84e0 2023-10-16T08:19:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80be7c0 con 0x7f38f80b84e0 2023-10-16T08:19:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f8154f40 con 0x7f38f80b84e0 2023-10-16T08:19:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80c40d0 con 0x7f38f80b84e0 2023-10-16T08:19:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.256+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f80c42c0 con 0x7f38f80b84e0 2023-10-16T08:19:01.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.260+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f38e80b4400 con 0x7f38f80b84e0 2023-10-16T08:19:01.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.260+0000 7f38eeffd640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38d80664d0 0x7f38d8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.260+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f38e811e520 con 0x7f38f80b84e0 2023-10-16T08:19:01.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.260+0000 7f38effff640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38d80664d0 0x7f38d8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.260+0000 7f38effff640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38d80664d0 0x7f38d8068990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f38f80d0fd0 tx=0x7f38e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.264+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f38e80eb7c0 con 0x7f38f80b84e0 2023-10-16T08:19:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.420+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f38f804e170 con 0x7f38f80b84e0 2023-10-16T08:19:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.420+0000 7f38eeffd640 1 -- 172.21.15.32:0/2223683863 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v54) v1 ==== 105+0+11 (secure 0 0 0) 0x7f38e80eede0 con 0x7f38f80b84e0 2023-10-16T08:19:01.424 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:19:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38d80664d0 msgr2=0x7f38d8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38d80664d0 0x7f38d8068990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f38f80d0fd0 tx=0x7f38e0009040 comp rx=0 tx=0).stop 2023-10-16T08:19:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 msgr2=0x7f38f80c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 0x7f38f80c59a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f38e80ac040 tx=0x7f38e801ad00 comp rx=0 tx=0).stop 2023-10-16T08:19:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 shutdown_connections 2023-10-16T08:19:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38f815cba0 0x7f38f814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f38d80664d0 0x7f38d8068990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f38f80d0210 0x7f38f81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 --2- 172.21.15.32:0/2223683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80b84e0 0x7f38f80c59a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 >> 172.21.15.32:0/2223683863 conn(0x7f38f80719c0 msgr2=0x7f38f815acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:01.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 shutdown_connections 2023-10-16T08:19:01.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.424+0000 7f38fcb1d640 1 -- 172.21.15.32:0/2223683863 wait complete. 2023-10-16T08:19:01.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:19:01.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 Processor -- start 2023-10-16T08:19:01.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 -- start start 2023-10-16T08:19:01.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23641597c0 0x7f23640ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2364159ba0 0x7f23640cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23640723d0 con 0x7f23640adc10 2023-10-16T08:19:01.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23640717c0 con 0x7f23641597c0 2023-10-16T08:19:01.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f236a59d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2364071aa0 con 0x7f2364159ba0 2023-10-16T08:19:01.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23641597c0 0x7f23640ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23641597c0 0x7f23640ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43790/0 (socket says 172.21.15.32:43790) 2023-10-16T08:19:01.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 -- 172.21.15.32:0/776064358 learned_addr learned my addr 172.21.15.32:0/776064358 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:01.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f23637fe640 1 --2- 172.21.15.32:0/776064358 >> v2:172.21.15.186:3300/0 conn(0x7f2364159ba0 0x7f23640cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 -- 172.21.15.32:0/776064358 >> v2:172.21.15.186:3300/0 conn(0x7f2364159ba0 msgr2=0x7f23640cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:01.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 --2- 172.21.15.32:0/776064358 >> v2:172.21.15.186:3300/0 conn(0x7f2364159ba0 0x7f23640cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 -- 172.21.15.32:0/776064358 >> v1:172.21.15.32:6789/0 conn(0x7f23640adc10 legacy=0x7f23640cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f2363fff640 1 -- 172.21.15.32:0/776064358 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364071d80 con 0x7f23641597c0 2023-10-16T08:19:01.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.588+0000 7f23637fe640 1 --2- 172.21.15.32:0/776064358 >> v2:172.21.15.186:3300/0 conn(0x7f2364159ba0 0x7f23640cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:01.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f2363fff640 1 --2- 172.21.15.32:0/776064358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23641597c0 0x7f23640ad6d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f23640b4690 tx=0x7f235401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc116fd08c1f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:01.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f2362ffd640 1 -- 172.21.15.32:0/776064358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540a6040 con 0x7f23641597c0 2023-10-16T08:19:01.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f2362ffd640 1 -- 172.21.15.32:0/776064358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23540aa600 con 0x7f23641597c0 2023-10-16T08:19:01.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f2362ffd640 1 -- 172.21.15.32:0/776064358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540aa990 con 0x7f23641597c0 2023-10-16T08:19:01.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 -- 172.21.15.32:0/776064358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23641597c0 msgr2=0x7f23640ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 --2- 172.21.15.32:0/776064358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23641597c0 0x7f23640ad6d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f23640b4690 tx=0x7f235401c500 comp rx=0 tx=0).stop 2023-10-16T08:19:01.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 -- 172.21.15.32:0/776064358 shutdown_connections 2023-10-16T08:19:01.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 --2- 172.21.15.32:0/776064358 >> v2:172.21.15.186:3300/0 conn(0x7f2364159ba0 0x7f23640cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 --2- 172.21.15.32:0/776064358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23641597c0 0x7f23640ad6d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 -- 172.21.15.32:0/776064358 >> 172.21.15.32:0/776064358 conn(0x7f236406c190 msgr2=0x7f236406c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:01.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 -- 172.21.15.32:0/776064358 shutdown_connections 2023-10-16T08:19:01.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 -- 172.21.15.32:0/776064358 wait complete. 2023-10-16T08:19:01.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.592+0000 7f236a59d640 1 Processor -- start 2023-10-16T08:19:01.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- start start 2023-10-16T08:19:01.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 0x7f23640c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23641597c0 0x7f2364157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2364159ba0 0x7f23641f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23641f5e20 con 0x7f23640adc10 2023-10-16T08:19:01.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23641f6100 con 0x7f2364159ba0 2023-10-16T08:19:01.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2364157620 con 0x7f23641597c0 2023-10-16T08:19:01.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 0x7f23640c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 0x7f23640c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43792/0 (socket says 172.21.15.32:43792) 2023-10-16T08:19:01.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 -- 172.21.15.32:0/2155763585 learned_addr learned my addr 172.21.15.32:0/2155763585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:01.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f23637fe640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23641597c0 0x7f2364157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2368b3a640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2364159ba0 0x7f23641f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 -- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23641597c0 msgr2=0x7f2364157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23641597c0 0x7f2364157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 -- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2364159ba0 msgr2=0x7f23641f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2364159ba0 0x7f23641f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 -- 172.21.15.32:0/2155763585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23540094d0 con 0x7f23640adc10 2023-10-16T08:19:01.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2363fff640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 0x7f23640c8160 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f235401a130 tx=0x7f235401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:01.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540b3800 con 0x7f23640adc10 2023-10-16T08:19:01.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23540b3960 con 0x7f23640adc10 2023-10-16T08:19:01.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540b3cd0 con 0x7f23640adc10 2023-10-16T08:19:01.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2364157900 con 0x7f23640adc10 2023-10-16T08:19:01.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2364157240 con 0x7f23640adc10 2023-10-16T08:19:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.596+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2364157b20 con 0x7f23640adc10 2023-10-16T08:19:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.600+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23540b2380 con 0x7f23640adc10 2023-10-16T08:19:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.600+0000 7f2360ff9640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23500664a0 0x7f2350068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.600+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f235411e150 con 0x7f23640adc10 2023-10-16T08:19:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.600+0000 7f23637fe640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23500664a0 0x7f2350068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.600+0000 7f23637fe640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23500664a0 0x7f2350068960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f23641546f0 tx=0x7f2358008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.604+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23540eb5a0 con 0x7f23640adc10 2023-10-16T08:19:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2364157430 con 0x7f23640adc10 2023-10-16T08:19:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f2360ff9640 1 -- 172.21.15.32:0/2155763585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f23540eebc0 con 0x7f23640adc10 2023-10-16T08:19:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23500664a0 msgr2=0x7f2350068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23500664a0 0x7f2350068960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f23641546f0 tx=0x7f2358008040 comp rx=0 tx=0).stop 2023-10-16T08:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 msgr2=0x7f23640c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 0x7f23640c8160 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f235401a130 tx=0x7f235401ad40 comp rx=0 tx=0).stop 2023-10-16T08:19:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 shutdown_connections 2023-10-16T08:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2364159ba0 0x7f23641f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f23500664a0 0x7f2350068960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f23641597c0 0x7f2364157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 --2- 172.21.15.32:0/2155763585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23640adc10 0x7f23640c8160 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 >> 172.21.15.32:0/2155763585 conn(0x7f236406c190 msgr2=0x7f236415c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 shutdown_connections 2023-10-16T08:19:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:01.772+0000 7f236a59d640 1 -- 172.21.15.32:0/2155763585 wait complete. 2023-10-16T08:19:01.795 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:19:01.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:01.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:01.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[50936]: fuse finished with error 0 and tester_r 0 2023-10-16T08:19:01.817 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:01.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:19:01.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:01.903 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:19:01.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:19:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 Processor -- start 2023-10-16T08:19:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 -- start start 2023-10-16T08:19:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 0x7f36c00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f36c015e4b0 0x7f36c00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c015ae10 con 0x7f36c00cfb80 2023-10-16T08:19:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c00cb9e0 con 0x7f36c015e4b0 2023-10-16T08:19:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c76c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b3ad0 con 0x7f36c015aa30 2023-10-16T08:19:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c5465640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 0x7f36c00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c5465640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 0x7f36c00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:43254/0 (socket says 172.21.15.32:43254) 2023-10-16T08:19:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c4c64640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f36c015e4b0 0x7f36c00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c5c66640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f36c00cfb80 0x7f36c00cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:37536/0 (socket says 172.21.15.32:37536) 2023-10-16T08:19:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.100+0000 7f36c5465640 1 -- 172.21.15.32:0/718265356 learned_addr learned my addr 172.21.15.32:0/718265356 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36c4c64640 1 -- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 msgr2=0x7f36c00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36c4c64640 1 --2- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 0x7f36c00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36c4c64640 1 -- 172.21.15.32:0/718265356 >> v1:172.21.15.186:6789/0 conn(0x7f36c00cfb80 legacy=0x7f36c00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36c4c64640 1 -- 172.21.15.32:0/718265356 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c015ea30 con 0x7f36c015e4b0 2023-10-16T08:19:02.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36b7fff640 1 -- 172.21.15.32:0/718265356 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33804302 0 0) 0x7f36b80014a0 con 0x7f36c00cfb80 2023-10-16T08:19:02.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36c5465640 1 --2- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 0x7f36c00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:02.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36c4c64640 1 --2- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015e4b0 0x7f36c00b2750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36b00097c0 tx=0x7f36b001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6f3e6b391c40a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:02.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36b7fff640 1 -- 172.21.15.32:0/718265356 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b00ab600 con 0x7f36c015e4b0 2023-10-16T08:19:02.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36b7fff640 1 -- 172.21.15.32:0/718265356 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b00a7050 con 0x7f36c015e4b0 2023-10-16T08:19:02.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.104+0000 7f36b7fff640 1 -- 172.21.15.32:0/718265356 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b00ab980 con 0x7f36c015e4b0 2023-10-16T08:19:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015e4b0 msgr2=0x7f36c00b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 --2- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015e4b0 0x7f36c00b2750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36b00097c0 tx=0x7f36b001c3c0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- 172.21.15.32:0/718265356 shutdown_connections 2023-10-16T08:19:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 --2- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015e4b0 0x7f36c00b2750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 --2- 172.21.15.32:0/718265356 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c015aa30 0x7f36c00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- 172.21.15.32:0/718265356 >> 172.21.15.32:0/718265356 conn(0x7f36c00cd1c0 msgr2=0x7f36c00cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- 172.21.15.32:0/718265356 shutdown_connections 2023-10-16T08:19:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- 172.21.15.32:0/718265356 wait complete. 2023-10-16T08:19:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 Processor -- start 2023-10-16T08:19:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- start start 2023-10-16T08:19:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c00cfb80 0x7f36c00b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015aa30 0x7f36c00ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 0x7f36c0154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b5390 con 0x7f36c015e4b0 2023-10-16T08:19:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c0155460 con 0x7f36c015aa30 2023-10-16T08:19:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c76c9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b47c0 con 0x7f36c00cfb80 2023-10-16T08:19:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 0x7f36c0154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 0x7f36c0154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43806/0 (socket says 172.21.15.32:43806) 2023-10-16T08:19:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5465640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c00cfb80 0x7f36c00b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 -- 172.21.15.32:0/1973625637 learned_addr learned my addr 172.21.15.32:0/1973625637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c4c64640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015aa30 0x7f36c00ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 -- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c00cfb80 msgr2=0x7f36c00b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c00cfb80 0x7f36c00b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 -- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015aa30 msgr2=0x7f36c00ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015aa30 0x7f36c00ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.108+0000 7f36c5c66640 1 -- 172.21.15.32:0/1973625637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc0094d0 con 0x7f36c015e4b0 2023-10-16T08:19:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36c5c66640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 0x7f36c0154f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f36b800c7e0 tx=0x7f36b800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800a640 con 0x7f36c015e4b0 2023-10-16T08:19:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b800a7c0 con 0x7f36c015e4b0 2023-10-16T08:19:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800abb0 con 0x7f36c015e4b0 2023-10-16T08:19:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b0009420 con 0x7f36c015e4b0 2023-10-16T08:19:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c00b4d40 con 0x7f36c015e4b0 2023-10-16T08:19:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c00ce730 con 0x7f36c015e4b0 2023-10-16T08:19:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f36b800ad10 con 0x7f36c015e4b0 2023-10-16T08:19:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36b5ffb640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f36a40664a0 0x7f36a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.112+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f36b80834a0 con 0x7f36c015e4b0 2023-10-16T08:19:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.116+0000 7f36c5465640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f36a40664a0 0x7f36a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.116+0000 7f36c5465640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f36a40664a0 0x7f36a4068960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f36c00c3c00 tx=0x7f36bc009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.116+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f36b80507d0 con 0x7f36c015e4b0 2023-10-16T08:19:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.308+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f36c0074010 con 0x7f36c015e4b0 2023-10-16T08:19:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.308+0000 7f36b5ffb640 1 -- 172.21.15.32:0/1973625637 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f36b8018070 con 0x7f36c015e4b0 2023-10-16T08:19:02.313 INFO:teuthology.orchestra.run.smithi032.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T08:19:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f36a40664a0 msgr2=0x7f36a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f36a40664a0 0x7f36a4068960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f36c00c3c00 tx=0x7f36bc009240 comp rx=0 tx=0).stop 2023-10-16T08:19:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 msgr2=0x7f36c0154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 0x7f36c0154f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f36b800c7e0 tx=0x7f36b800cca0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 shutdown_connections 2023-10-16T08:19:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c015e4b0 0x7f36c0154f20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c015aa30 0x7f36c00ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f36a40664a0 0x7f36a4068960 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 --2- 172.21.15.32:0/1973625637 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f36c00cfb80 0x7f36c00b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 >> 172.21.15.32:0/1973625637 conn(0x7f36c00cd1c0 msgr2=0x7f36c00d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:02.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 shutdown_connections 2023-10-16T08:19:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:02.312+0000 7f36c76c9640 1 -- 172.21.15.32:0/1973625637 wait complete. 2023-10-16T08:19:02.333 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:19:02.333 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:19:02.334 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:19:02.334 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:19:02.334 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:02.335 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:19:02.335 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:19:02.335 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:19:02.336 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:19:02.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.337 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:19:02.347 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:19:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:19:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:19:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:19:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.348 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:19:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:19:02.349 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:19:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:19:02.350 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:19:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:19:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:19:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42263sec preferred_lft 42263sec 2023-10-16T08:19:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:19:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.352 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:19:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:19:02.353 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:19:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:19:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.355 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:19:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:19:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:19:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.356 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:19:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:19:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.358 INFO:teuthology.orchestra.run.smithi032.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:19:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T08:19:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:02.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.359 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:19:02.407 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:19:02.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.410 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:19:02.466 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:19:02.467 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:19:02.467 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:19:02.467 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:19:02.468 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:19:02.468 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:19:02.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.469 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:19:02.469 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:19:02.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:19:02.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:19:02.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:19:02.471 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:19:02.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.563 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:19:02.573 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:19:02.574 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:19:02.574 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:19:02.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:19:02.638 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:19:02.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:19:02.639 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:02.693 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:19:02.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:19:02.694 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:02.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:19:02.810 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:19:02.867 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:19:02.867 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:19:02.867 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:19:02.868 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:19:02.868 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:19:02.868 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:19:02.868 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:19:02.869 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:19:02.869 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:19:02.869 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:19:02.870 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:19:02.870 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:19:02.870 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:19:02.870 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:19:02.871 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:19:02.871 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:19:02.871 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:19:02.872 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:19:02.872 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:19:02.872 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:19:02.872 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:19:02.872 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:19:02.873 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:19:02.873 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:19:02.873 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:19:02.873 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:19:02.873 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:19:02.874 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:19:02.874 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:19:02.874 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:02.874 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:02.875 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:02.875 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:02.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:02.876 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:19:02.925 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:19:02.925 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:19:02.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:19:02.998 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:19:03.051 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:19:03.052 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:19:03.052 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:19:03.052 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:19:03.053 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:19:03.053 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:19:03.053 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:19:03.053 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:19:03.054 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:19:03.054 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:19:03.054 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:19:03.054 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:19:03.055 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:19:03.055 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:19:03.055 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:19:03.055 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:19:03.055 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:19:03.056 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:19:03.056 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:19:03.056 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:19:03.056 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:19:03.056 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:19:03.057 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:19:03.057 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:19:03.057 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:19:03.057 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:19:03.057 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:19:03.058 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:19:03.058 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:19:03.058 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:03.058 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:03.058 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:03.059 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:03.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:03.060 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:19:03.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:19:03.072+0000 7f782dc44580 -1 init, newargv = 0x557f80558f20 newargc=15 2023-10-16T08:19:03.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51334]: starting ceph client 2023-10-16T08:19:03.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51334]: starting fuse 2023-10-16T08:19:03.085 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:19:03.085 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:19:03.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:19:03.086 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:19:03.086 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:19:03.087 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:19:03.087 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:19:03.087 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:03.088 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:19:03.088 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:19:03.089 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:19:03.089 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:19:03.090 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:03.091 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:19:03.091 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:19:03.091 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:19:03.092 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:19:03.092 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:03.092 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:19:03.093 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:19:03.093 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:19:03.094 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:19:03.094 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:19:03.094 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:19:03.095 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:19:03.095 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:19:03.096 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:03.096 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:19:03.096 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:19:03.213 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.51334.asok 2023-10-16T08:19:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:19:03.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51334.asok 2023-10-16T08:19:03.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:03.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51334.asok status 2023-10-16T08:19:03.398 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:19:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:19:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:19:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:19:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:19:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:19:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "51334", 2023-10-16T08:19:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:19:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:19:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:19:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6444, 2023-10-16T08:19:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:19:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:19:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:19:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 6444 2023-10-16T08:19:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:19:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:19:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:19:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 424022291 2023-10-16T08:19:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:19:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:19:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:19:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:19:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 424022291 2023-10-16T08:19:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.6444 192.168.0.1:0/424022291", 2023-10-16T08:19:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/424022291", 2023-10-16T08:19:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:19:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 28, 2023-10-16T08:19:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 54, 2023-10-16T08:19:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:19:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:19:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1" 2023-10-16T08:19:03.406 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:03.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:03.413 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:19:03.414 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:03.424 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:19:03.425 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:03.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:03.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:03.489 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2023-10-16T08:19:03.490 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:19:03.490 DEBUG:teuthology.orchestra.run.smithi032:> import errno 2023-10-16T08:19:03.490 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:03.491 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-10-16T08:19:03.491 DEBUG:teuthology.orchestra.run.smithi032:> try: 2023-10-16T08:19:03.491 DEBUG:teuthology.orchestra.run.smithi032:> if False: 2023-10-16T08:19:03.492 DEBUG:teuthology.orchestra.run.smithi032:> ret = os.read(fd, 1024) 2023-10-16T08:19:03.492 DEBUG:teuthology.orchestra.run.smithi032:> else: 2023-10-16T08:19:03.492 DEBUG:teuthology.orchestra.run.smithi032:> os.write(fd, b'"'"'content'"'"') 2023-10-16T08:19:03.493 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2023-10-16T08:19:03.493 DEBUG:teuthology.orchestra.run.smithi032:> if e.errno != errno.EPERM: 2023-10-16T08:19:03.493 DEBUG:teuthology.orchestra.run.smithi032:> raise 2023-10-16T08:19:03.494 DEBUG:teuthology.orchestra.run.smithi032:> else: 2023-10-16T08:19:03.494 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("client does not check permission of data pool") 2023-10-16T08:19:03.494 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:19:03.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:19:03.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=testcephfs1_data' 2023-10-16T08:19:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 Processor -- start 2023-10-16T08:19:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 -- start start 2023-10-16T08:19:03.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0b89b0 0x7f577c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f577c0d0130 0x7f577c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c161720 con 0x7f577c0b85d0 2023-10-16T08:19:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c0d05a0 con 0x7f577c0d0130 2023-10-16T08:19:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5781409640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c160ba0 con 0x7f577c0b89b0 2023-10-16T08:19:03.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f577affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f577c0b85d0 0x7f577c15dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53798/0 (socket says 172.21.15.32:53798) 2023-10-16T08:19:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f577affd640 1 -- 172.21.15.32:0/2870552344 learned_addr learned my addr 172.21.15.32:0/2870552344 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f577b7fe640 1 --2- 172.21.15.32:0/2870552344 >> v2:172.21.15.186:3300/0 conn(0x7f577c0d0130 0x7f577c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.800+0000 7f5773fff640 1 --2- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0b89b0 0x7f577c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577a7fc640 1 -- 172.21.15.32:0/2870552344 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789498015 0 0) 0x7f577c161720 con 0x7f577c0b85d0 2023-10-16T08:19:03.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577a7fc640 1 -- 172.21.15.32:0/2870552344 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5768003530 con 0x7f577c0b85d0 2023-10-16T08:19:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577a7fc640 1 -- 172.21.15.32:0/2870552344 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169283351 0 0) 0x7f5768003530 con 0x7f577c0b85d0 2023-10-16T08:19:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577b7fe640 1 -- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0b89b0 msgr2=0x7f577c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577b7fe640 1 --2- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0b89b0 0x7f577c0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577b7fe640 1 -- 172.21.15.32:0/2870552344 >> v1:172.21.15.32:6789/0 conn(0x7f577c0b85d0 legacy=0x7f577c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577b7fe640 1 -- 172.21.15.32:0/2870552344 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c160e80 con 0x7f577c0d0130 2023-10-16T08:19:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577b7fe640 1 --2- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0d0130 0x7f577c0d0e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5774009d90 tx=0x7f577401c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d082b6792ad8d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577a7fc640 1 -- 172.21.15.32:0/2870552344 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c001680 con 0x7f577c0d0130 2023-10-16T08:19:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577a7fc640 1 -- 172.21.15.32:0/2870552344 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740a5640 con 0x7f577c0d0130 2023-10-16T08:19:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.804+0000 7f577a7fc640 1 -- 172.21.15.32:0/2870552344 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740a59d0 con 0x7f577c0d0130 2023-10-16T08:19:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0d0130 msgr2=0x7f577c0d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 --2- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0d0130 0x7f577c0d0e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5774009d90 tx=0x7f577401c840 comp rx=0 tx=0).stop 2023-10-16T08:19:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- 172.21.15.32:0/2870552344 shutdown_connections 2023-10-16T08:19:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 --2- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0d0130 0x7f577c0d0e00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 --2- 172.21.15.32:0/2870552344 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0b89b0 0x7f577c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- 172.21.15.32:0/2870552344 >> 172.21.15.32:0/2870552344 conn(0x7f577c06b750 msgr2=0x7f577c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- 172.21.15.32:0/2870552344 shutdown_connections 2023-10-16T08:19:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- 172.21.15.32:0/2870552344 wait complete. 2023-10-16T08:19:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 Processor -- start 2023-10-16T08:19:03.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- start start 2023-10-16T08:19:03.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0b85d0 0x7f577c0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:03.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 0x7f577c14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:03.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0d0130 0x7f577c14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c14fc90 con 0x7f577c0b89b0 2023-10-16T08:19:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c160040 con 0x7f577c0b85d0 2023-10-16T08:19:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5781409640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c15f4c0 con 0x7f577c0d0130 2023-10-16T08:19:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 0x7f577c14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 0x7f577c14edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34518/0 (socket says 172.21.15.32:34518) 2023-10-16T08:19:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 -- 172.21.15.32:0/1787209943 learned_addr learned my addr 172.21.15.32:0/1787209943 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f577affd640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0b85d0 0x7f577c0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f577b7fe640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0d0130 0x7f577c14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 -- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0d0130 msgr2=0x7f577c14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0d0130 0x7f577c14f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 -- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0b85d0 msgr2=0x7f577c0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0b85d0 0x7f577c0c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.808+0000 7f5773fff640 1 -- 172.21.15.32:0/1787209943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774009a70 con 0x7f577c0b89b0 2023-10-16T08:19:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f5773fff640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 0x7f577c14edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f576400e900 tx=0x7f576400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764009690 con 0x7f577c0b89b0 2023-10-16T08:19:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576400ca00 con 0x7f577c0b89b0 2023-10-16T08:19:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576400cdb0 con 0x7f577c0b89b0 2023-10-16T08:19:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c15f7a0 con 0x7f577c0b89b0 2023-10-16T08:19:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c15fd60 con 0x7f577c0b89b0 2023-10-16T08:19:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c0c6710 con 0x7f577c0b89b0 2023-10-16T08:19:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5764013470 con 0x7f577c0b89b0 2023-10-16T08:19:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.812+0000 7f57737fe640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f575c0664a0 0x7f575c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.816+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f57640841a0 con 0x7f577c0b89b0 2023-10-16T08:19:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.816+0000 7f577affd640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f575c0664a0 0x7f575c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.816+0000 7f577affd640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f575c0664a0 0x7f575c068960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f576c001ba0 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:03.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:03.820+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5764051550 con 0x7f577c0b89b0 2023-10-16T08:19:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.032+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=testcephfs1_data"]} v 0) v1 -- 0x7f577c04e170 con 0x7f577c0b89b0 2023-10-16T08:19:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.032+0000 7f57737fe640 1 -- 172.21.15.32:0/1787209943 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=testcephfs1_data"]}]=0 updated caps for client.0 v15) v1 ==== 188+0+145 (secure 0 0 0) 0x7f5764054b70 con 0x7f577c0b89b0 2023-10-16T08:19:04.037 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:19:04.037 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:19:04.038 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-10-16T08:19:04.038 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:19:04.038 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow w pool=testcephfs1_data" 2023-10-16T08:19:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:19:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f575c0664a0 msgr2=0x7f575c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f575c0664a0 0x7f575c068960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f576c001ba0 tx=0x7f576c009040 comp rx=0 tx=0).stop 2023-10-16T08:19:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 msgr2=0x7f577c14edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 0x7f577c14edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f576400e900 tx=0x7f576400edc0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 shutdown_connections 2023-10-16T08:19:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f575c0664a0 0x7f575c068960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f577c0d0130 0x7f577c14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c0b89b0 0x7f577c14edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 --2- 172.21.15.32:0/1787209943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c0b85d0 0x7f577c0c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 >> 172.21.15.32:0/1787209943 conn(0x7f577c06b750 msgr2=0x7f577c0b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 shutdown_connections 2023-10-16T08:19:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.036+0000 7f5781409640 1 -- 172.21.15.32:0/1787209943 wait complete. 2023-10-16T08:19:04.058 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:04.059 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:04.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 Processor -- start 2023-10-16T08:19:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 -- start start 2023-10-16T08:19:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03415cb90 0x7fc0340cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc0340d15d0 0x7fc0340c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0340ce770 con 0x7fc0340d3010 2023-10-16T08:19:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340d0600 con 0x7fc03415cb90 2023-10-16T08:19:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03c580640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340c6450 con 0x7fc0340d15d0 2023-10-16T08:19:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03415cb90 0x7fc0340cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03415cb90 0x7fc0340cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34522/0 (socket says 172.21.15.32:34522) 2023-10-16T08:19:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 -- 172.21.15.32:0/2651463874 learned_addr learned my addr 172.21.15.32:0/2651463874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03ab1d640 1 --2- 172.21.15.32:0/2651463874 >> v2:172.21.15.186:3300/0 conn(0x7fc0340d15d0 0x7fc0340c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 -- 172.21.15.32:0/2651463874 >> v2:172.21.15.186:3300/0 conn(0x7fc0340d15d0 msgr2=0x7fc0340c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 --2- 172.21.15.32:0/2651463874 >> v2:172.21.15.186:3300/0 conn(0x7fc0340d15d0 0x7fc0340c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 -- 172.21.15.32:0/2651463874 >> v1:172.21.15.186:6789/0 conn(0x7fc0340d3010 legacy=0x7fc0340d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.252+0000 7fc03a31c640 1 -- 172.21.15.32:0/2651463874 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340d08a0 con 0x7fc03415cb90 2023-10-16T08:19:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03a31c640 1 --2- 172.21.15.32:0/2651463874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc0340cfb10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc03408ba70 tx=0x7fc03001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ffde2142a3cbe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03931a640 1 -- 172.21.15.32:0/2651463874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300ac620 con 0x7fc03415cb90 2023-10-16T08:19:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03931a640 1 -- 172.21.15.32:0/2651463874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300ac780 con 0x7fc03415cb90 2023-10-16T08:19:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03931a640 1 -- 172.21.15.32:0/2651463874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300acb10 con 0x7fc03415cb90 2023-10-16T08:19:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 -- 172.21.15.32:0/2651463874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 msgr2=0x7fc0340cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 --2- 172.21.15.32:0/2651463874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc0340cfb10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc03408ba70 tx=0x7fc03001c3c0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 -- 172.21.15.32:0/2651463874 shutdown_connections 2023-10-16T08:19:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 --2- 172.21.15.32:0/2651463874 >> v2:172.21.15.186:3300/0 conn(0x7fc0340d15d0 0x7fc0340c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 --2- 172.21.15.32:0/2651463874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc0340cfb10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 -- 172.21.15.32:0/2651463874 >> 172.21.15.32:0/2651463874 conn(0x7fc03409bc50 msgr2=0x7fc0340d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 -- 172.21.15.32:0/2651463874 shutdown_connections 2023-10-16T08:19:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.256+0000 7fc03c580640 1 -- 172.21.15.32:0/2651463874 wait complete. 2023-10-16T08:19:04.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 Processor -- start 2023-10-16T08:19:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- start start 2023-10-16T08:19:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0340d15d0 0x7fc034141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0340d3010 0x7fc034143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc034158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034155b20 con 0x7fc03415cb90 2023-10-16T08:19:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034142270 con 0x7fc0340d3010 2023-10-16T08:19:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc034142550 con 0x7fc0340d15d0 2023-10-16T08:19:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc034158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc034158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34536/0 (socket says 172.21.15.32:34536) 2023-10-16T08:19:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 -- 172.21.15.32:0/1116213199 learned_addr learned my addr 172.21.15.32:0/1116213199 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03a31c640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0340d15d0 0x7fc034141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc039b1b640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0340d3010 0x7fc034143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 -- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0340d15d0 msgr2=0x7fc034141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0340d15d0 0x7fc034141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 -- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0340d3010 msgr2=0x7fc034143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0340d3010 0x7fc034143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 -- 172.21.15.32:0/1116213199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030009480 con 0x7fc03415cb90 2023-10-16T08:19:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03ab1d640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc034158580 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc02c007e40 tx=0x7fc02c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c014620 con 0x7fc03415cb90 2023-10-16T08:19:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00ee90 con 0x7fc03415cb90 2023-10-16T08:19:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc034142830 con 0x7fc03415cb90 2023-10-16T08:19:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0149e0 con 0x7fc03415cb90 2023-10-16T08:19:04.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc034142df0 con 0x7fc03415cb90 2023-10-16T08:19:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.260+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc034074010 con 0x7fc03415cb90 2023-10-16T08:19:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.264+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc02c014b40 con 0x7fc03415cb90 2023-10-16T08:19:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.264+0000 7fc02affd640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0180664a0 0x7fc018068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.264+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fc02c0848d0 con 0x7fc03415cb90 2023-10-16T08:19:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.264+0000 7fc03a31c640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0180664a0 0x7fc018068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.264+0000 7fc03a31c640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0180664a0 0x7fc018068960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc0300ae000 tx=0x7fc0300a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.268+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc02c051c00 con 0x7fc03415cb90 2023-10-16T08:19:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.424+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc03406b360 con 0x7fc03415cb90 2023-10-16T08:19:04.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc02affd640 1 -- 172.21.15.32:0/1116213199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fc02c055220 con 0x7fc03415cb90 2023-10-16T08:19:04.430 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:04.430 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:04.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0180664a0 msgr2=0x7fc018068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0180664a0 0x7fc018068960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc0300ae000 tx=0x7fc0300a7000 comp rx=0 tx=0).stop 2023-10-16T08:19:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 msgr2=0x7fc034158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc034158580 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc02c007e40 tx=0x7fc02c00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 shutdown_connections 2023-10-16T08:19:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415cb90 0x7fc034158580 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0340d3010 0x7fc034143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc0180664a0 0x7fc018068960 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 --2- 172.21.15.32:0/1116213199 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc0340d15d0 0x7fc034141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 >> 172.21.15.32:0/1116213199 conn(0x7fc03409bc50 msgr2=0x7fc0340b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 shutdown_connections 2023-10-16T08:19:04.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.428+0000 7fc03c580640 1 -- 172.21.15.32:0/1116213199 wait complete. 2023-10-16T08:19:04.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:18:48.142496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:18:48.637534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:19:04.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:19:04.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 Processor -- start 2023-10-16T08:19:04.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 -- start start 2023-10-16T08:19:04.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d8406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d8406a5a0 0x7f5d840c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840b4360 con 0x7f5d840b3f00 2023-10-16T08:19:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d840cd740 con 0x7f5d8406a5a0 2023-10-16T08:19:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d893b8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840b4600 con 0x7f5d84159900 2023-10-16T08:19:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d8406a5a0 0x7f5d840c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d8406a5a0 0x7f5d840c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34538/0 (socket says 172.21.15.32:34538) 2023-10-16T08:19:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 -- 172.21.15.32:0/2451275808 learned_addr learned my addr 172.21.15.32:0/2451275808 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d82ffd640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d8406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 -- 172.21.15.32:0/2451275808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 msgr2=0x7f5d8406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d8406e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 -- 172.21.15.32:0/2451275808 >> v1:172.21.15.32:6789/0 conn(0x7f5d840b3f00 legacy=0x7f5d84069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d837fe640 1 -- 172.21.15.32:0/2451275808 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d840c8dd0 con 0x7f5d8406a5a0 2023-10-16T08:19:04.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.596+0000 7f5d82ffd640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d8406e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:04.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d837fe640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d840c9b90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009e50 tx=0x7f5d7c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1dfc6aecd91bd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d827fc640 1 -- 172.21.15.32:0/2451275808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0a58a0 con 0x7f5d8406a5a0 2023-10-16T08:19:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d827fc640 1 -- 172.21.15.32:0/2451275808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0a5a00 con 0x7f5d8406a5a0 2023-10-16T08:19:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d827fc640 1 -- 172.21.15.32:0/2451275808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0a5d90 con 0x7f5d8406a5a0 2023-10-16T08:19:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 -- 172.21.15.32:0/2451275808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 msgr2=0x7f5d840c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d840c9b90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009e50 tx=0x7f5d7c01c9e0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 -- 172.21.15.32:0/2451275808 shutdown_connections 2023-10-16T08:19:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d840c9b90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 --2- 172.21.15.32:0/2451275808 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d8406e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 -- 172.21.15.32:0/2451275808 >> 172.21.15.32:0/2451275808 conn(0x7f5d840b0da0 msgr2=0x7f5d840ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 -- 172.21.15.32:0/2451275808 shutdown_connections 2023-10-16T08:19:04.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.600+0000 7f5d893b8640 1 -- 172.21.15.32:0/2451275808 wait complete. 2023-10-16T08:19:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 Processor -- start 2023-10-16T08:19:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 -- start start 2023-10-16T08:19:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d8414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d840b3f00 0x7f5d840abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d840b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840ad570 con 0x7f5d8406a5a0 2023-10-16T08:19:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840ac4d0 con 0x7f5d840b3f00 2023-10-16T08:19:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840ac7b0 con 0x7f5d84159900 2023-10-16T08:19:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d8414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d8414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34552/0 (socket says 172.21.15.32:34552) 2023-10-16T08:19:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 -- 172.21.15.32:0/389810726 learned_addr learned my addr 172.21.15.32:0/389810726 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d7bfff640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d840b3f00 0x7f5d840abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d837fe640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d840b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 -- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 msgr2=0x7f5d840b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d840b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 -- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d840b3f00 msgr2=0x7f5d840abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d840b3f00 0x7f5d840abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 -- 172.21.15.32:0/389810726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009b30 con 0x7f5d8406a5a0 2023-10-16T08:19:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d82ffd640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d8414eea0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400d800 tx=0x7f5d7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740096c0 con 0x7f5d8406a5a0 2023-10-16T08:19:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74009820 con 0x7f5d8406a5a0 2023-10-16T08:19:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74009bd0 con 0x7f5d8406a5a0 2023-10-16T08:19:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d840aca90 con 0x7f5d8406a5a0 2023-10-16T08:19:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.604+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d840ad050 con 0x7f5d8406a5a0 2023-10-16T08:19:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.608+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d840cecc0 con 0x7f5d8406a5a0 2023-10-16T08:19:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.608+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5d74009d30 con 0x7f5d8406a5a0 2023-10-16T08:19:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.608+0000 7f5d7b7fe640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5d640664a0 0x7f5d64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.608+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f5d74084770 con 0x7f5d8406a5a0 2023-10-16T08:19:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.608+0000 7f5d7bfff640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5d640664a0 0x7f5d64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.608+0000 7f5d7bfff640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5d640664a0 0x7f5d64068960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d84158ed0 tx=0x7f5d6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.612+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5d74051a90 con 0x7f5d8406a5a0 2023-10-16T08:19:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.772+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d8404e170 con 0x7f5d8406a5a0 2023-10-16T08:19:04.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.772+0000 7f5d7b7fe640 1 -- 172.21.15.32:0/389810726 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d740550b0 con 0x7f5d8406a5a0 2023-10-16T08:19:04.776 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:19:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5d640664a0 msgr2=0x7f5d64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5d640664a0 0x7f5d64068960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d84158ed0 tx=0x7f5d6c009040 comp rx=0 tx=0).stop 2023-10-16T08:19:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 msgr2=0x7f5d8414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d8414eea0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400d800 tx=0x7f5d7400dcc0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 shutdown_connections 2023-10-16T08:19:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f5d640664a0 0x7f5d64068960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f5d84159900 0x7f5d840b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d840b3f00 0x7f5d840abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 --2- 172.21.15.32:0/389810726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8406a5a0 0x7f5d8414eea0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 >> 172.21.15.32:0/389810726 conn(0x7f5d840b0da0 msgr2=0x7f5d84155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 shutdown_connections 2023-10-16T08:19:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.776+0000 7f5d893b8640 1 -- 172.21.15.32:0/389810726 wait complete. 2023-10-16T08:19:04.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:19:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.940+0000 7fbbf5a6b640 1 Processor -- start 2023-10-16T08:19:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.940+0000 7fbbf5a6b640 1 -- start start 2023-10-16T08:19:04.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.940+0000 7fbbf5a6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbbf015cb60 0x7fbbf00c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.940+0000 7fbbf5a6b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf006ac10 con 0x7fbbf006f560 2023-10-16T08:19:04.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.940+0000 7fbbf5a6b640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf0072110 con 0x7fbbf015cf40 2023-10-16T08:19:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.940+0000 7fbbf5a6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf00723f0 con 0x7fbbf015cb60 2023-10-16T08:19:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbbf006f560 0x7fbbf0071400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53810/0 (socket says 172.21.15.32:53810) 2023-10-16T08:19:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeffff640 1 -- 172.21.15.32:0/2488732441 learned_addr learned my addr 172.21.15.32:0/2488732441 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbef7fe640 1 --2- 172.21.15.32:0/2488732441 >> v2:172.21.15.186:3300/0 conn(0x7fbbf015cb60 0x7fbbf00c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094624405 0 0) 0x7fbbf006ac10 con 0x7fbbf006f560 2023-10-16T08:19:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd8003530 con 0x7fbbf006f560 2023-10-16T08:19:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475297144 0 0) 0x7fbbd8003530 con 0x7fbbf006f560 2023-10-16T08:19:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 >> v2:172.21.15.186:3300/0 conn(0x7fbbf015cb60 msgr2=0x7fbbf00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 --2- 172.21.15.32:0/2488732441 >> v2:172.21.15.186:3300/0 conn(0x7fbbf015cb60 0x7fbbf00c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 >> v1:172.21.15.186:6789/0 conn(0x7fbbf015cf40 legacy=0x7fbbf006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf006a5c0 con 0x7fbbf006f560 2023-10-16T08:19:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fbbe8001a00 con 0x7fbbf006f560 2023-10-16T08:19:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 >> v1:172.21.15.32:6789/0 conn(0x7fbbf006f560 legacy=0x7fbbf0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd8003980 0x7fbbd8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd80047d0 0x7fbbd8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd8005190 0x7fbbd800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00723f0 con 0x7fbbd80047d0 2023-10-16T08:19:04.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd800c030 con 0x7fbbd8005190 2023-10-16T08:19:04.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd800c1f0 con 0x7fbbd8003980 2023-10-16T08:19:04.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeeffd640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd80047d0 0x7fbbd8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeffff640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd8005190 0x7fbbd800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbef7fe640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd8003980 0x7fbbd8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeeffd640 1 -- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd8003980 msgr2=0x7fbbd8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeeffd640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd8003980 0x7fbbd8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeeffd640 1 -- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd8005190 msgr2=0x7fbbd800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeeffd640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbd8005190 0x7fbbd800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeffff640 1 -- 172.21.15.32:0/2488732441 reap_dead start 2023-10-16T08:19:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.944+0000 7fbbeeffd640 1 -- 172.21.15.32:0/2488732441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8003530 con 0x7fbbd80047d0 2023-10-16T08:19:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbeeffd640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd80047d0 0x7fbbd8004c50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4001780 tx=0x7fbbe401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d09116708892965 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40a4ea0 con 0x7fbbd80047d0 2023-10-16T08:19:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe40a7050 con 0x7fbbd80047d0 2023-10-16T08:19:04.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbee7fc640 1 -- 172.21.15.32:0/2488732441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4003600 con 0x7fbbd80047d0 2023-10-16T08:19:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/2488732441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd80047d0 msgr2=0x7fbbd8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd80047d0 0x7fbbd8004c50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4001780 tx=0x7fbbe401c3e0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/2488732441 shutdown_connections 2023-10-16T08:19:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/2488732441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd80047d0 0x7fbbd8004c50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/2488732441 >> 172.21.15.32:0/2488732441 conn(0x7fbbf00cf7a0 msgr2=0x7fbbf00cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:04.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/2488732441 shutdown_connections 2023-10-16T08:19:04.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/2488732441 wait complete. 2023-10-16T08:19:04.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 Processor -- start 2023-10-16T08:19:04.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.948+0000 7fbbf5a6b640 1 -- start start 2023-10-16T08:19:04.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd80047d0 0x7fbbf00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf0159210 0x7fbbf01f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 0x7fbbf0160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf01f31c0 con 0x7fbbf01f26f0 2023-10-16T08:19:04.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf01f2c00 con 0x7fbbf0159210 2023-10-16T08:19:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf01f2ee0 con 0x7fbbd80047d0 2023-10-16T08:19:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 0x7fbbf0160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 0x7fbbf0160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34566/0 (socket says 172.21.15.32:34566) 2023-10-16T08:19:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 -- 172.21.15.32:0/1299134611 learned_addr learned my addr 172.21.15.32:0/1299134611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbef7fe640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd80047d0 0x7fbbf00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeeffd640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf0159210 0x7fbbf01f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 -- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd80047d0 msgr2=0x7fbbf00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd80047d0 0x7fbbf00c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 -- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf0159210 msgr2=0x7fbbf01f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf0159210 0x7fbbf01f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 -- 172.21.15.32:0/1299134611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00c6a10 con 0x7fbbf01f26f0 2023-10-16T08:19:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbeffff640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 0x7fbbf0160db0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8001570 tx=0x7fbbe80115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8007ad0 con 0x7fbbf01f26f0 2023-10-16T08:19:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8004b70 con 0x7fbbf01f26f0 2023-10-16T08:19:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf00c6cf0 con 0x7fbbf01f26f0 2023-10-16T08:19:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8006110 con 0x7fbbf01f26f0 2023-10-16T08:19:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf00aba30 con 0x7fbbf01f26f0 2023-10-16T08:19:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf00c6870 con 0x7fbbf01f26f0 2023-10-16T08:19:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.952+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbbe8006270 con 0x7fbbf01f26f0 2023-10-16T08:19:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.956+0000 7fbbd7fff640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbdc0664a0 0x7fbbdc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.956+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fbbe8086f70 con 0x7fbbf01f26f0 2023-10-16T08:19:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.956+0000 7fbbef7fe640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbdc0664a0 0x7fbbdc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.956+0000 7fbbef7fe640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbdc0664a0 0x7fbbdc068960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbbf015add0 tx=0x7fbbe0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:04.960+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbbf00c6870 con 0x7fbbf01f26f0 2023-10-16T08:19:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.116+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbf00c6870 con 0x7fbbf01f26f0 2023-10-16T08:19:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.116+0000 7fbbd7fff640 1 -- 172.21.15.32:0/1299134611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v54) v1 ==== 109+0+11 (secure 0 0 0) 0x7fbbf00c6870 con 0x7fbbf01f26f0 2023-10-16T08:19:05.120 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:19:05.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbdc0664a0 msgr2=0x7fbbdc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbdc0664a0 0x7fbbdc068960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbbf015add0 tx=0x7fbbe0008040 comp rx=0 tx=0).stop 2023-10-16T08:19:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 msgr2=0x7fbbf0160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 0x7fbbf0160db0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8001570 tx=0x7fbbe80115c0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 shutdown_connections 2023-10-16T08:19:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf01f26f0 0x7fbbf0160db0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbf0159210 0x7fbbf01f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbbdc0664a0 0x7fbbdc068960 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 --2- 172.21.15.32:0/1299134611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbbd80047d0 0x7fbbf00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 >> 172.21.15.32:0/1299134611 conn(0x7fbbf00cf7a0 msgr2=0x7fbbf00cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:05.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 shutdown_connections 2023-10-16T08:19:05.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.120+0000 7fbbf5a6b640 1 -- 172.21.15.32:0/1299134611 wait complete. 2023-10-16T08:19:05.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:19:05.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.284+0000 7f8399365640 1 Processor -- start 2023-10-16T08:19:05.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.284+0000 7f8399365640 1 -- start start 2023-10-16T08:19:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8399365640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f839415ca10 0x7f83940ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8399365640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 0x7f83940d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8399365640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83940cd770 con 0x7f83940d13c0 2023-10-16T08:19:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8399365640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83940d1870 con 0x7f839415ca10 2023-10-16T08:19:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8399365640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83940cda50 con 0x7f839415cdf0 2023-10-16T08:19:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 0x7f83940d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8392ffd640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f839415ca10 0x7f83940ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 0x7f83940d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:48202/0 (socket says 172.21.15.32:48202) 2023-10-16T08:19:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 -- 172.21.15.32:0/165943092 learned_addr learned my addr 172.21.15.32:0/165943092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83937fe640 1 --1- 172.21.15.32:0/165943092 >> v1:172.21.15.186:6789/0 conn(0x7f83940d13c0 0x7f83940cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:58480/0 (socket says 172.21.15.32:58480) 2023-10-16T08:19:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 -- 172.21.15.32:0/165943092 >> v2:172.21.15.186:3300/0 conn(0x7f839415ca10 msgr2=0x7f83940ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 --2- 172.21.15.32:0/165943092 >> v2:172.21.15.186:3300/0 conn(0x7f839415ca10 0x7f83940ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 -- 172.21.15.32:0/165943092 >> v1:172.21.15.186:6789/0 conn(0x7f83940d13c0 legacy=0x7f83940cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f83927fc640 1 -- 172.21.15.32:0/165943092 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83940cdd30 con 0x7f839415cdf0 2023-10-16T08:19:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.288+0000 7f8392ffd640 1 --2- 172.21.15.32:0/165943092 >> v2:172.21.15.186:3300/0 conn(0x7f839415ca10 0x7f83940ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f83927fc640 1 --2- 172.21.15.32:0/165943092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 0x7f83940d0e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8388009870 tx=0x7f838801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8f6eeed33c240c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8391ffb640 1 -- 172.21.15.32:0/165943092 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83880ab620 con 0x7f839415cdf0 2023-10-16T08:19:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8391ffb640 1 -- 172.21.15.32:0/165943092 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83880ab780 con 0x7f839415cdf0 2023-10-16T08:19:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8391ffb640 1 -- 172.21.15.32:0/165943092 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83880abb10 con 0x7f839415cdf0 2023-10-16T08:19:05.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 -- 172.21.15.32:0/165943092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 msgr2=0x7f83940d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 --2- 172.21.15.32:0/165943092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 0x7f83940d0e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8388009870 tx=0x7f838801c510 comp rx=0 tx=0).stop 2023-10-16T08:19:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 -- 172.21.15.32:0/165943092 shutdown_connections 2023-10-16T08:19:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 --2- 172.21.15.32:0/165943092 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415cdf0 0x7f83940d0e80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 --2- 172.21.15.32:0/165943092 >> v2:172.21.15.186:3300/0 conn(0x7f839415ca10 0x7f83940ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 -- 172.21.15.32:0/165943092 >> 172.21.15.32:0/165943092 conn(0x7f839409bb90 msgr2=0x7f83940d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 -- 172.21.15.32:0/165943092 shutdown_connections 2023-10-16T08:19:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 -- 172.21.15.32:0/165943092 wait complete. 2023-10-16T08:19:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 Processor -- start 2023-10-16T08:19:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.292+0000 7f8399365640 1 -- start start 2023-10-16T08:19:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 0x7f83940b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415ca10 0x7f83940b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839415cdf0 0x7f83940c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940b73f0 con 0x7f83940d13c0 2023-10-16T08:19:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940c97c0 con 0x7f839415cdf0 2023-10-16T08:19:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8394155180 con 0x7f839415ca10 2023-10-16T08:19:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 0x7f83940b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 0x7f83940b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34580/0 (socket says 172.21.15.32:34580) 2023-10-16T08:19:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 -- 172.21.15.32:0/1495190097 learned_addr learned my addr 172.21.15.32:0/1495190097 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f83937fe640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839415cdf0 0x7f83940c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f83927fc640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415ca10 0x7f83940b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 -- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415ca10 msgr2=0x7f83940b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415ca10 0x7f83940b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 -- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839415cdf0 msgr2=0x7f83940c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839415cdf0 0x7f83940c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 -- 172.21.15.32:0/1495190097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840093b0 con 0x7f83940d13c0 2023-10-16T08:19:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8392ffd640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 0x7f83940b3c00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f838400ebe0 tx=0x7f838400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384009760 con 0x7f83940d13c0 2023-10-16T08:19:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838400cc30 con 0x7f83940d13c0 2023-10-16T08:19:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400eee0 con 0x7f83940d13c0 2023-10-16T08:19:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83880094d0 con 0x7f83940d13c0 2023-10-16T08:19:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8394158540 con 0x7f83940d13c0 2023-10-16T08:19:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.296+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83940b7660 con 0x7f83940d13c0 2023-10-16T08:19:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.300+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f83840059a0 con 0x7f83940d13c0 2023-10-16T08:19:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.300+0000 7f837b7fe640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83800664a0 0x7f8380068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.300+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f8384084370 con 0x7f83940d13c0 2023-10-16T08:19:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.300+0000 7f83927fc640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83800664a0 0x7f8380068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.300+0000 7f83927fc640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83800664a0 0x7f8380068960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f83880af000 tx=0x7f83880a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.304+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f83840517d0 con 0x7f83940d13c0 2023-10-16T08:19:05.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f8394067230 con 0x7f83940d13c0 2023-10-16T08:19:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f837b7fe640 1 -- 172.21.15.32:0/1495190097 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v54) v1 ==== 105+0+11 (secure 0 0 0) 0x7f8384020020 con 0x7f83940d13c0 2023-10-16T08:19:05.466 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:19:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83800664a0 msgr2=0x7f8380068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83800664a0 0x7f8380068960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f83880af000 tx=0x7f83880a7000 comp rx=0 tx=0).stop 2023-10-16T08:19:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 msgr2=0x7f83940b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 0x7f83940b3c00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f838400ebe0 tx=0x7f838400c520 comp rx=0 tx=0).stop 2023-10-16T08:19:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 shutdown_connections 2023-10-16T08:19:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f839415cdf0 0x7f83940c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f83800664a0 0x7f8380068960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f839415ca10 0x7f83940b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 --2- 172.21.15.32:0/1495190097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83940d13c0 0x7f83940b3c00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 >> 172.21.15.32:0/1495190097 conn(0x7f839409bb90 msgr2=0x7f839415bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 shutdown_connections 2023-10-16T08:19:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.464+0000 7f8399365640 1 -- 172.21.15.32:0/1495190097 wait complete. 2023-10-16T08:19:05.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:19:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 Processor -- start 2023-10-16T08:19:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 -- start start 2023-10-16T08:19:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f287c070c40 0x7f287c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b4020 0x7f287c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287c15be60 con 0x7f287c0b0620 2023-10-16T08:19:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c15b3b0 con 0x7f287c070c40 2023-10-16T08:19:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2882854640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0b0a90 con 0x7f287c0b4020 2023-10-16T08:19:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2880df1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f287c0b0620 0x7f287c15c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53816/0 (socket says 172.21.15.32:53816) 2023-10-16T08:19:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f2880df1640 1 -- 172.21.15.32:0/1504428227 learned_addr learned my addr 172.21.15.32:0/1504428227 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287bfff640 1 --2- 172.21.15.32:0/1504428227 >> v2:172.21.15.186:3300/0 conn(0x7f287c070c40 0x7f287c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287b7fe640 1 --2- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b4020 0x7f287c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287affd640 1 -- 172.21.15.32:0/1504428227 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919763782 0 0) 0x7f287c15be60 con 0x7f287c0b0620 2023-10-16T08:19:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287affd640 1 -- 172.21.15.32:0/1504428227 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2864003530 con 0x7f287c0b0620 2023-10-16T08:19:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287affd640 1 -- 172.21.15.32:0/1504428227 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244470488 0 0) 0x7f2864003530 con 0x7f287c0b0620 2023-10-16T08:19:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287bfff640 1 -- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b4020 msgr2=0x7f287c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287bfff640 1 --2- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b4020 0x7f287c0b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287bfff640 1 -- 172.21.15.32:0/1504428227 >> v1:172.21.15.32:6789/0 conn(0x7f287c0b0620 legacy=0x7f287c15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.632+0000 7f287bfff640 1 -- 172.21.15.32:0/1504428227 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c15c1f0 con 0x7f287c070c40 2023-10-16T08:19:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f287bfff640 1 --2- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c15ae70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f286c0097c0 tx=0x7f286c01c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a32199a35c5020 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f287affd640 1 -- 172.21.15.32:0/1504428227 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874001a00 con 0x7f287c070c40 2023-10-16T08:19:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f287affd640 1 -- 172.21.15.32:0/1504428227 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c0a7050 con 0x7f287c070c40 2023-10-16T08:19:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f287affd640 1 -- 172.21.15.32:0/1504428227 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0ab620 con 0x7f287c070c40 2023-10-16T08:19:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 -- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 msgr2=0x7f287c15ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 --2- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c15ae70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f286c0097c0 tx=0x7f286c01c310 comp rx=0 tx=0).stop 2023-10-16T08:19:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 -- 172.21.15.32:0/1504428227 shutdown_connections 2023-10-16T08:19:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 --2- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b4020 0x7f287c0b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 --2- 172.21.15.32:0/1504428227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c15ae70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 -- 172.21.15.32:0/1504428227 >> 172.21.15.32:0/1504428227 conn(0x7f287c06c190 msgr2=0x7f287c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 -- 172.21.15.32:0/1504428227 shutdown_connections 2023-10-16T08:19:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.636+0000 7f2882854640 1 -- 172.21.15.32:0/1504428227 wait complete. 2023-10-16T08:19:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 Processor -- start 2023-10-16T08:19:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 -- start start 2023-10-16T08:19:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b0620 0x7f287c0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 0x7f287c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c153970 con 0x7f287c0b4020 2023-10-16T08:19:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c4a60 con 0x7f287c070c40 2023-10-16T08:19:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c152cf0 con 0x7f287c0b0620 2023-10-16T08:19:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 0x7f287c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 0x7f287c0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34584/0 (socket says 172.21.15.32:34584) 2023-10-16T08:19:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 -- 172.21.15.32:0/3411863346 learned_addr learned my addr 172.21.15.32:0/3411863346 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f287bfff640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f287b7fe640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b0620 0x7f287c0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 -- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b0620 msgr2=0x7f287c0c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b0620 0x7f287c0c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 -- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 msgr2=0x7f287c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 -- 172.21.15.32:0/3411863346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c009420 con 0x7f287c0b4020 2023-10-16T08:19:05.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f287b7fe640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b0620 0x7f287c0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:05.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2880df1640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 0x7f287c0b1db0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f287400cc50 tx=0x7f287400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:05.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400ce60 con 0x7f287c0b4020 2023-10-16T08:19:05.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874012400 con 0x7f287c0b4020 2023-10-16T08:19:05.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c152fd0 con 0x7f287c0b4020 2023-10-16T08:19:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28740127b0 con 0x7f287c0b4020 2023-10-16T08:19:05.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.640+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c153590 con 0x7f287c0b4020 2023-10-16T08:19:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.644+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c15bb60 con 0x7f287c0b4020 2023-10-16T08:19:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.644+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2874012910 con 0x7f287c0b4020 2023-10-16T08:19:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.644+0000 7f2878ff9640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28680664a0 0x7f2868068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:05.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.644+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f28740846a0 con 0x7f287c0b4020 2023-10-16T08:19:05.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.644+0000 7f287bfff640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28680664a0 0x7f2868068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:05.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.644+0000 7f287bfff640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28680664a0 0x7f2868068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f286c0ad000 tx=0x7f286c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:05.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.648+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f287c15bb60 con 0x7f287c0b4020 2023-10-16T08:19:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f287c15bb30 con 0x7f287c0b4020 2023-10-16T08:19:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2878ff9640 1 -- 172.21.15.32:0/3411863346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28740519b0 con 0x7f287c0b4020 2023-10-16T08:19:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:19:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28680664a0 msgr2=0x7f2868068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28680664a0 0x7f2868068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f286c0ad000 tx=0x7f286c0a6000 comp rx=0 tx=0).stop 2023-10-16T08:19:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 msgr2=0x7f287c0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 0x7f287c0b1db0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f287400cc50 tx=0x7f287400a430 comp rx=0 tx=0).stop 2023-10-16T08:19:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 shutdown_connections 2023-10-16T08:19:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c0b4020 0x7f287c0b1db0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f28680664a0 0x7f2868068960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f287c0b0620 0x7f287c0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 --2- 172.21.15.32:0/3411863346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f287c070c40 0x7f287c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 >> 172.21.15.32:0/3411863346 conn(0x7f287c06c190 msgr2=0x7f287c14e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 shutdown_connections 2023-10-16T08:19:05.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:05.820+0000 7f2882854640 1 -- 172.21.15.32:0/3411863346 wait complete. 2023-10-16T08:19:05.843 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:05.844 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:05.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29bdeea640 1 Processor -- start 2023-10-16T08:19:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29bdeea640 1 -- start start 2023-10-16T08:19:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29bdeea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29b815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29bdeea640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b815be60 con 0x7f29b80b07f0 2023-10-16T08:19:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29bdeea640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b815c1f0 con 0x7f29b80b4020 2023-10-16T08:19:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29bdeea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b815b3b0 con 0x7f29b8070c40 2023-10-16T08:19:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29b80b07f0 0x7f29b8164490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53822/0 (socket says 172.21.15.32:53822) 2023-10-16T08:19:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b7fff640 1 -- 172.21.15.32:0/1124095428 learned_addr learned my addr 172.21.15.32:0/1124095428 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b77fe640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29b815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222104581 0 0) 0x7f29b815be60 con 0x7f29b80b07f0 2023-10-16T08:19:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a0003530 con 0x7f29b80b07f0 2023-10-16T08:19:06.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238074032 0 0) 0x7f29b815c1f0 con 0x7f29b80b4020 2023-10-16T08:19:06.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b815be60 con 0x7f29b80b4020 2023-10-16T08:19:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2968708643 0 0) 0x7f29a0003530 con 0x7f29b80b07f0 2023-10-16T08:19:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 msgr2=0x7f29b815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29b815ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 >> v1:172.21.15.186:6789/0 conn(0x7f29b80b4020 legacy=0x7f29b80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8071020 con 0x7f29b80b07f0 2023-10-16T08:19:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f29b0001a00 con 0x7f29b80b07f0 2023-10-16T08:19:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b77fe640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29b815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 >> v1:172.21.15.32:6789/0 conn(0x7f29b80b07f0 legacy=0x7f29b8164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0003b40 0x7f29a0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a00047d0 0x7f29a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b815b3b0 con 0x7f29a0003b40 2023-10-16T08:19:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a000ba70 con 0x7f29a00047d0 2023-10-16T08:19:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a000bc30 con 0x7f29b8070c40 2023-10-16T08:19:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.040+0000 7f29b77fe640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0003b40 0x7f29a0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b6ffd640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a00047d0 0x7f29a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b7fff640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b77fe640 1 -- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 msgr2=0x7f29a0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b77fe640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29a0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b77fe640 1 -- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a00047d0 msgr2=0x7f29a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b77fe640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29a00047d0 0x7f29a0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b77fe640 1 -- 172.21.15.32:0/1124095428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8009420 con 0x7f29a0003b40 2023-10-16T08:19:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b7fff640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29b8070c40 0x7f29a0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b77fe640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0003b40 0x7f29a0003f40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009c80 tx=0x7f29a801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691caeb78fb41b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80a5830 con 0x7f29a0003b40 2023-10-16T08:19:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a80a5990 con 0x7f29a0003b40 2023-10-16T08:19:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29b67fc640 1 -- 172.21.15.32:0/1124095428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80a5d40 con 0x7f29a0003b40 2023-10-16T08:19:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29bdeea640 1 -- 172.21.15.32:0/1124095428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0003b40 msgr2=0x7f29a0003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.044+0000 7f29bdeea640 1 --2- 172.21.15.32:0/1124095428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a0003b40 0x7f29a0003f40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009c80 tx=0x7f29a801cb90 comp rx=0 tx=0).stop 2023-10-16T08:19:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 -- 172.21.15.32:0/1124095428 reap_dead start 2023-10-16T08:19:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- 172.21.15.32:0/1124095428 shutdown_connections 2023-10-16T08:19:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- 172.21.15.32:0/1124095428 >> 172.21.15.32:0/1124095428 conn(0x7f29b806c190 msgr2=0x7f29b806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- 172.21.15.32:0/1124095428 shutdown_connections 2023-10-16T08:19:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- 172.21.15.32:0/1124095428 wait complete. 2023-10-16T08:19:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 Processor -- start 2023-10-16T08:19:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- start start 2023-10-16T08:19:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a0003b40 0x7f29b80b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29b81510f0 0x7f29b80c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 0x7f29b8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80c0e00 con 0x7f29b80b28d0 2023-10-16T08:19:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b8151570 con 0x7f29b81510f0 2023-10-16T08:19:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29bdeea640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80b1830 con 0x7f29a0003b40 2023-10-16T08:19:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 0x7f29b8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 0x7f29b8159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34600/0 (socket says 172.21.15.32:34600) 2023-10-16T08:19:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 -- 172.21.15.32:0/3212314611 learned_addr learned my addr 172.21.15.32:0/3212314611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b6ffd640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29b81510f0 0x7f29b80c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b77fe640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a0003b40 0x7f29b80b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 -- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a0003b40 msgr2=0x7f29b80b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a0003b40 0x7f29b80b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 -- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29b81510f0 msgr2=0x7f29b80c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29b81510f0 0x7f29b80c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.048+0000 7f29b7fff640 1 -- 172.21.15.32:0/3212314611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80b1b10 con 0x7f29b80b28d0 2023-10-16T08:19:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f29b7fff640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 0x7f29b8159940 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f29a0004750 tx=0x7f29b0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000e8c0 con 0x7f29b80b28d0 2023-10-16T08:19:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b000ea20 con 0x7f29b80b28d0 2023-10-16T08:19:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000edd0 con 0x7f29b80b28d0 2023-10-16T08:19:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80b1df0 con 0x7f29b80b28d0 2023-10-16T08:19:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b80b23b0 con 0x7f29b80b28d0 2023-10-16T08:19:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b8074010 con 0x7f29b80b28d0 2023-10-16T08:19:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29b001d460 con 0x7f29b80b28d0 2023-10-16T08:19:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f299ffff640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29a40664d0 0x7f29a4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f29b00840e0 con 0x7f29b80b28d0 2023-10-16T08:19:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.052+0000 7f29b77fe640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29a40664d0 0x7f29a4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.056+0000 7f29b77fe640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29a40664d0 0x7f29a4068990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f29a80af000 tx=0x7f29a80046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.056+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29b0051360 con 0x7f29b80b28d0 2023-10-16T08:19:06.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.216+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29b80cde50 con 0x7f29b80b28d0 2023-10-16T08:19:06.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.216+0000 7f299ffff640 1 -- 172.21.15.32:0/3212314611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f29b0054980 con 0x7f29b80b28d0 2023-10-16T08:19:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:06.219 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:06.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29a40664d0 msgr2=0x7f29a4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29a40664d0 0x7f29a4068990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f29a80af000 tx=0x7f29a80046d0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 msgr2=0x7f29b8159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 0x7f29b8159940 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f29a0004750 tx=0x7f29b0007a30 comp rx=0 tx=0).stop 2023-10-16T08:19:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 shutdown_connections 2023-10-16T08:19:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29b80b28d0 0x7f29b8159940 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29b81510f0 0x7f29b80c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f29a40664d0 0x7f29a4068990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 --2- 172.21.15.32:0/3212314611 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f29a0003b40 0x7f29b80b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 >> 172.21.15.32:0/3212314611 conn(0x7f29b806c190 msgr2=0x7f29b80adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 shutdown_connections 2023-10-16T08:19:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.220+0000 7f29bdeea640 1 -- 172.21.15.32:0/3212314611 wait complete. 2023-10-16T08:19:06.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:18:48.142496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:18:48.637534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:19:06.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:19:06.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 Processor -- start 2023-10-16T08:19:06.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 -- start start 2023-10-16T08:19:06.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f2f600d0210 0x7f2f600d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f600b84e0 0x7f2f600b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f600d1670 con 0x7f2f6015cba0 2023-10-16T08:19:06.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600d1460 con 0x7f2f600b84e0 2023-10-16T08:19:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f64a54640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600d1950 con 0x7f2f600d0210 2023-10-16T08:19:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f5ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f600b84e0 0x7f2f600b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f5ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f600b84e0 0x7f2f600b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34616/0 (socket says 172.21.15.32:34616) 2023-10-16T08:19:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.388+0000 7f2f5ed9d640 1 -- 172.21.15.32:0/1678173158 learned_addr learned my addr 172.21.15.32:0/1678173158 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:06.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f57fff640 1 --2- 172.21.15.32:0/1678173158 >> v2:172.21.15.186:3300/0 conn(0x7f2f600d0210 0x7f2f600d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5ed9d640 1 -- 172.21.15.32:0/1678173158 >> v2:172.21.15.186:3300/0 conn(0x7f2f600d0210 msgr2=0x7f2f600d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5ed9d640 1 --2- 172.21.15.32:0/1678173158 >> v2:172.21.15.186:3300/0 conn(0x7f2f600d0210 0x7f2f600d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5ed9d640 1 -- 172.21.15.32:0/1678173158 >> v1:172.21.15.186:6789/0 conn(0x7f2f6015cba0 legacy=0x7f2f600cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5ed9d640 1 -- 172.21.15.32:0/1678173158 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600d1c30 con 0x7f2f600b84e0 2023-10-16T08:19:06.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5ed9d640 1 --2- 172.21.15.32:0/1678173158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600b84e0 0x7f2f600b5cf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f600cf010 tx=0x7f2f5801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39559301dcf742b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5dd9b640 1 -- 172.21.15.32:0/1678173158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580a5800 con 0x7f2f600b84e0 2023-10-16T08:19:06.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5dd9b640 1 -- 172.21.15.32:0/1678173158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f580a5960 con 0x7f2f600b84e0 2023-10-16T08:19:06.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.392+0000 7f2f5dd9b640 1 -- 172.21.15.32:0/1678173158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580a5cf0 con 0x7f2f600b84e0 2023-10-16T08:19:06.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- 172.21.15.32:0/1678173158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600b84e0 msgr2=0x7f2f600b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 --2- 172.21.15.32:0/1678173158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600b84e0 0x7f2f600b5cf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f600cf010 tx=0x7f2f5801c940 comp rx=0 tx=0).stop 2023-10-16T08:19:06.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- 172.21.15.32:0/1678173158 shutdown_connections 2023-10-16T08:19:06.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 --2- 172.21.15.32:0/1678173158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600b84e0 0x7f2f600b5cf0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 --2- 172.21.15.32:0/1678173158 >> v2:172.21.15.186:3300/0 conn(0x7f2f600d0210 0x7f2f600d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- 172.21.15.32:0/1678173158 >> 172.21.15.32:0/1678173158 conn(0x7f2f600719c0 msgr2=0x7f2f60071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:06.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- 172.21.15.32:0/1678173158 shutdown_connections 2023-10-16T08:19:06.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- 172.21.15.32:0/1678173158 wait complete. 2023-10-16T08:19:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 Processor -- start 2023-10-16T08:19:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- start start 2023-10-16T08:19:06.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f600b84e0 0x7f2f60148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 0x7f2f60148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f6015cba0 0x7f2f60149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60148b50 con 0x7f2f600d0210 2023-10-16T08:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600c6770 con 0x7f2f600b84e0 2023-10-16T08:19:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f64a54640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60156f40 con 0x7f2f6015cba0 2023-10-16T08:19:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 0x7f2f60148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 0x7f2f60148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34626/0 (socket says 172.21.15.32:34626) 2023-10-16T08:19:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 -- 172.21.15.32:0/3184354125 learned_addr learned my addr 172.21.15.32:0/3184354125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f5e59c640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f600b84e0 0x7f2f60148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f5ed9d640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f6015cba0 0x7f2f60149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 -- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f6015cba0 msgr2=0x7f2f60149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f6015cba0 0x7f2f60149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 -- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f600b84e0 msgr2=0x7f2f60148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f600b84e0 0x7f2f60148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.396+0000 7f2f57fff640 1 -- 172.21.15.32:0/3184354125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f58009b30 con 0x7f2f600d0210 2023-10-16T08:19:06.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f57fff640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 0x7f2f60148cd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800d7d0 tx=0x7f2f4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48018070 con 0x7f2f600d0210 2023-10-16T08:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48004020 con 0x7f2f600d0210 2023-10-16T08:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f480041d0 con 0x7f2f600d0210 2023-10-16T08:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f601566a0 con 0x7f2f600d0210 2023-10-16T08:19:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f60156c60 con 0x7f2f600d0210 2023-10-16T08:19:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f600b6030 con 0x7f2f600d0210 2023-10-16T08:19:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2f48004330 con 0x7f2f600d0210 2023-10-16T08:19:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f56ffd640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2f400664a0 0x7f2f40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f2f48083ca0 con 0x7f2f600d0210 2023-10-16T08:19:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.400+0000 7f2f5e59c640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2f400664a0 0x7f2f40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.404+0000 7f2f5e59c640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2f400664a0 0x7f2f40068960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2f50007fd0 tx=0x7f2f50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.404+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2f48051100 con 0x7f2f600d0210 2023-10-16T08:19:06.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.560+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f6004e170 con 0x7f2f600d0210 2023-10-16T08:19:06.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f56ffd640 1 -- 172.21.15.32:0/3184354125 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f48054720 con 0x7f2f600d0210 2023-10-16T08:19:06.566 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:19:06.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2f400664a0 msgr2=0x7f2f40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2f400664a0 0x7f2f40068960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2f50007fd0 tx=0x7f2f50009040 comp rx=0 tx=0).stop 2023-10-16T08:19:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 msgr2=0x7f2f60148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 0x7f2f60148cd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800d7d0 tx=0x7f2f4800dc90 comp rx=0 tx=0).stop 2023-10-16T08:19:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 shutdown_connections 2023-10-16T08:19:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2f400664a0 0x7f2f40068960 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2f6015cba0 0x7f2f60149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f600d0210 0x7f2f60148cd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 --2- 172.21.15.32:0/3184354125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f600b84e0 0x7f2f60148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 >> 172.21.15.32:0/3184354125 conn(0x7f2f600719c0 msgr2=0x7f2f60153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:06.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 shutdown_connections 2023-10-16T08:19:06.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.564+0000 7f2f64a54640 1 -- 172.21.15.32:0/3184354125 wait complete. 2023-10-16T08:19:06.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:19:06.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.728+0000 7f1d06446640 1 Processor -- start 2023-10-16T08:19:06.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.728+0000 7f1d06446640 1 -- start start 2023-10-16T08:19:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d06446640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d0015cba0 0x7f1d000d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d06446640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1d000d0210 0x7f1d000b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d06446640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d000d1670 con 0x7f1d000b84e0 2023-10-16T08:19:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d06446640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d000d1460 con 0x7f1d000d0210 2023-10-16T08:19:06.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d06446640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000d1950 con 0x7f1d0015cba0 2023-10-16T08:19:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1cff7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d000b84e0 0x7f1d000cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53836/0 (socket says 172.21.15.32:53836) 2023-10-16T08:19:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1cff7fe640 1 -- 172.21.15.32:0/3947412400 learned_addr learned my addr 172.21.15.32:0/3947412400 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d049e3640 1 --2- 172.21.15.32:0/3947412400 >> v2:172.21.15.186:3300/0 conn(0x7f1d000d0210 0x7f1d000b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1cfffff640 1 --2- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d0015cba0 0x7f1d000d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1cfeffd640 1 -- 172.21.15.32:0/3947412400 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017467760 0 0) 0x7f1d000d1670 con 0x7f1d000b84e0 2023-10-16T08:19:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1cfeffd640 1 -- 172.21.15.32:0/3947412400 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce8003530 con 0x7f1d000b84e0 2023-10-16T08:19:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d049e3640 1 -- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d0015cba0 msgr2=0x7f1d000d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d049e3640 1 --2- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d0015cba0 0x7f1d000d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d049e3640 1 -- 172.21.15.32:0/3947412400 >> v1:172.21.15.32:6789/0 conn(0x7f1d000b84e0 legacy=0x7f1d000cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.732+0000 7f1d049e3640 1 -- 172.21.15.32:0/3947412400 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000d1c30 con 0x7f1d000d0210 2023-10-16T08:19:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d049e3640 1 --2- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d000d0210 0x7f1d000b5cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009d90 tx=0x7f1cf801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7abb4a7d53a0ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1cfeffd640 1 -- 172.21.15.32:0/3947412400 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80a5a10 con 0x7f1d000d0210 2023-10-16T08:19:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1cfeffd640 1 -- 172.21.15.32:0/3947412400 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf80a5b70 con 0x7f1d000d0210 2023-10-16T08:19:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1cfeffd640 1 -- 172.21.15.32:0/3947412400 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80ad440 con 0x7f1d000d0210 2023-10-16T08:19:06.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 -- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d000d0210 msgr2=0x7f1d000b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 --2- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d000d0210 0x7f1d000b5cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009d90 tx=0x7f1cf801c950 comp rx=0 tx=0).stop 2023-10-16T08:19:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 -- 172.21.15.32:0/3947412400 shutdown_connections 2023-10-16T08:19:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 --2- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d000d0210 0x7f1d000b5cf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 --2- 172.21.15.32:0/3947412400 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d0015cba0 0x7f1d000d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 -- 172.21.15.32:0/3947412400 >> 172.21.15.32:0/3947412400 conn(0x7f1d000719c0 msgr2=0x7f1d00071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 -- 172.21.15.32:0/3947412400 shutdown_connections 2023-10-16T08:19:06.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 -- 172.21.15.32:0/3947412400 wait complete. 2023-10-16T08:19:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.736+0000 7f1d06446640 1 Processor -- start 2023-10-16T08:19:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- start start 2023-10-16T08:19:06.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 0x7f1d00149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d000d0210 0x7f1d001f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0015cba0 0x7f1d001f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00148ce0 con 0x7f1d000b84e0 2023-10-16T08:19:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cd170 con 0x7f1d0015cba0 2023-10-16T08:19:06.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00148a90 con 0x7f1d000d0210 2023-10-16T08:19:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 0x7f1d00149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 0x7f1d00149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34636/0 (socket says 172.21.15.32:34636) 2023-10-16T08:19:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 -- 172.21.15.32:0/2826621253 learned_addr learned my addr 172.21.15.32:0/2826621253 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d049e3640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0015cba0 0x7f1d001f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cff7fe640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d000d0210 0x7f1d001f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 -- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d000d0210 msgr2=0x7f1d001f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d000d0210 0x7f1d001f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 -- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0015cba0 msgr2=0x7f1d001f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0015cba0 0x7f1d001f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 -- 172.21.15.32:0/2826621253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8009a70 con 0x7f1d000b84e0 2023-10-16T08:19:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d049e3640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0015cba0 0x7f1d001f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfffff640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 0x7f1d00149670 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000e8a0 tx=0x7f1cf000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0009510 con 0x7f1d000b84e0 2023-10-16T08:19:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf000c940 con 0x7f1d000b84e0 2023-10-16T08:19:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000ccb0 con 0x7f1d000b84e0 2023-10-16T08:19:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d001486e0 con 0x7f1d000b84e0 2023-10-16T08:19:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d00149180 con 0x7f1d000b84e0 2023-10-16T08:19:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.740+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0004e170 con 0x7f1d000b84e0 2023-10-16T08:19:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.744+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1cf0013470 con 0x7f1d000b84e0 2023-10-16T08:19:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.744+0000 7f1cfcff9640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1cec0664a0 0x7f1cec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.744+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f1cf0084150 con 0x7f1d000b84e0 2023-10-16T08:19:06.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.744+0000 7f1cff7fe640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1cec0664a0 0x7f1cec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:06.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.744+0000 7f1cff7fe640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1cec0664a0 0x7f1cec068960 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1d00144fe0 tx=0x7f1cf4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:06.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.748+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1cf00513d0 con 0x7f1d000b84e0 2023-10-16T08:19:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d00148270 con 0x7f1d000b84e0 2023-10-16T08:19:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1cfcff9640 1 -- 172.21.15.32:0/2826621253 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v54) v1 ==== 109+0+11 (secure 0 0 0) 0x7f1cf00549f0 con 0x7f1d000b84e0 2023-10-16T08:19:06.907 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:19:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1cec0664a0 msgr2=0x7f1cec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1cec0664a0 0x7f1cec068960 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1d00144fe0 tx=0x7f1cf4009040 comp rx=0 tx=0).stop 2023-10-16T08:19:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 msgr2=0x7f1d00149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 0x7f1d00149670 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000e8a0 tx=0x7f1cf000eca0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 shutdown_connections 2023-10-16T08:19:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d0015cba0 0x7f1d001f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1cec0664a0 0x7f1cec068960 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1d000d0210 0x7f1d001f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 --2- 172.21.15.32:0/2826621253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d000b84e0 0x7f1d00149670 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.904+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 >> 172.21.15.32:0/2826621253 conn(0x7f1d000719c0 msgr2=0x7f1d00072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.908+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 shutdown_connections 2023-10-16T08:19:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:06.908+0000 7f1d06446640 1 -- 172.21.15.32:0/2826621253 wait complete. 2023-10-16T08:19:06.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:19:07.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 Processor -- start 2023-10-16T08:19:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 -- start start 2023-10-16T08:19:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f820806d010 0x7f82080c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 0x7f82080c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820815ae30 con 0x7f8208069e90 2023-10-16T08:19:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080cb090 con 0x7f820806d010 2023-10-16T08:19:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820c9c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820806d470 con 0x7f82080c26b0 2023-10-16T08:19:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8206d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 0x7f82080c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8206d9d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 0x7f82080c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:48270/0 (socket says 172.21.15.32:48270) 2023-10-16T08:19:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8206d9d640 1 -- 172.21.15.32:0/1705753691 learned_addr learned my addr 172.21.15.32:0/1705753691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820659c640 1 --1- 172.21.15.32:0/1705753691 >> v1:172.21.15.186:6789/0 conn(0x7f8208069e90 0x7f820806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:58498/0 (socket says 172.21.15.32:58498) 2023-10-16T08:19:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8205d9b640 1 --2- 172.21.15.32:0/1705753691 >> v2:172.21.15.186:3300/0 conn(0x7f820806d010 0x7f82080c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8205d9b640 1 -- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 msgr2=0x7f82080c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8205d9b640 1 --2- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 0x7f82080c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8205d9b640 1 -- 172.21.15.32:0/1705753691 >> v1:172.21.15.186:6789/0 conn(0x7f8208069e90 legacy=0x7f820806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8205d9b640 1 -- 172.21.15.32:0/1705753691 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080ca510 con 0x7f820806d010 2023-10-16T08:19:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f8206d9d640 1 --2- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 0x7f82080c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.072+0000 7f820559a640 1 -- 172.21.15.32:0/1705753691 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359388940 0 0) 0x7f820815ae30 con 0x7f8208069e90 2023-10-16T08:19:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.076+0000 7f8205d9b640 1 --2- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c9ce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0097c0 tx=0x7f81fc01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7612f53cd4fadbfe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.076+0000 7f820559a640 1 -- 172.21.15.32:0/1705753691 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0a8970 con 0x7f820806d010 2023-10-16T08:19:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.076+0000 7f820559a640 1 -- 172.21.15.32:0/1705753691 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc0a8ad0 con 0x7f820806d010 2023-10-16T08:19:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.076+0000 7f820559a640 1 -- 172.21.15.32:0/1705753691 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0a8e60 con 0x7f820806d010 2023-10-16T08:19:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.076+0000 7f820c9c4640 1 -- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 msgr2=0x7f82080c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.076+0000 7f820c9c4640 1 --2- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c9ce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0097c0 tx=0x7f81fc01ec70 comp rx=0 tx=0).stop 2023-10-16T08:19:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- 172.21.15.32:0/1705753691 shutdown_connections 2023-10-16T08:19:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 --2- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82080c26b0 0x7f82080c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 --2- 172.21.15.32:0/1705753691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c9ce0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- 172.21.15.32:0/1705753691 >> 172.21.15.32:0/1705753691 conn(0x7f82080b0da0 msgr2=0x7f82080ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- 172.21.15.32:0/1705753691 shutdown_connections 2023-10-16T08:19:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- 172.21.15.32:0/1705753691 wait complete. 2023-10-16T08:19:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 Processor -- start 2023-10-16T08:19:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- start start 2023-10-16T08:19:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8208069e90 0x7f82081578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 0x7f82080c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080c5e00 con 0x7f82080c26b0 2023-10-16T08:19:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080c8f40 con 0x7f820806d010 2023-10-16T08:19:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820c9c4640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8208157f30 con 0x7f8208069e90 2023-10-16T08:19:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 0x7f82080c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 0x7f82080c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34642/0 (socket says 172.21.15.32:34642) 2023-10-16T08:19:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 -- 172.21.15.32:0/3141935035 learned_addr learned my addr 172.21.15.32:0/3141935035 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820659c640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8208069e90 0x7f82081578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8205d9b640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 -- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8208069e90 msgr2=0x7f82081578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8208069e90 0x7f82081578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 -- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 msgr2=0x7f82080c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f8206d9d640 1 -- 172.21.15.32:0/3141935035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200009b30 con 0x7f82080c26b0 2023-10-16T08:19:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.080+0000 7f820659c640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8208069e90 0x7f82081578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f8206d9d640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 0x7f82080c8a00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8200009f10 tx=0x7f820000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200018070 con 0x7f82080c26b0 2023-10-16T08:19:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820000f040 con 0x7f82080c26b0 2023-10-16T08:19:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000ce60 con 0x7f82080c26b0 2023-10-16T08:19:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc009420 con 0x7f82080c26b0 2023-10-16T08:19:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82081584f0 con 0x7f82080c26b0 2023-10-16T08:19:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820804e170 con 0x7f82080c26b0 2023-10-16T08:19:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8200014480 con 0x7f82080c26b0 2023-10-16T08:19:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f81f6ffd640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e80664a0 0x7f81e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.084+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f8200084220 con 0x7f82080c26b0 2023-10-16T08:19:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.088+0000 7f820659c640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e80664a0 0x7f81e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.088+0000 7f820659c640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e80664a0 0x7f81e8068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f81f8000f70 tx=0x7f81f8007dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:07.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.092+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82000515d0 con 0x7f82080c26b0 2023-10-16T08:19:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f82080c2090 con 0x7f82080c26b0 2023-10-16T08:19:07.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f81f6ffd640 1 -- 172.21.15.32:0/3141935035 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v54) v1 ==== 105+0+11 (secure 0 0 0) 0x7f8200054bf0 con 0x7f82080c26b0 2023-10-16T08:19:07.251 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:19:07.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e80664a0 msgr2=0x7f81e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e80664a0 0x7f81e8068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f81f8000f70 tx=0x7f81f8007dd0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 msgr2=0x7f82080c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 0x7f82080c8a00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8200009f10 tx=0x7f820000c490 comp rx=0 tx=0).stop 2023-10-16T08:19:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 shutdown_connections 2023-10-16T08:19:07.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82080c26b0 0x7f82080c8a00 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f820806d010 0x7f82080c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e80664a0 0x7f81e8068960 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 --2- 172.21.15.32:0/3141935035 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8208069e90 0x7f82081578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 >> 172.21.15.32:0/3141935035 conn(0x7f82080b0da0 msgr2=0x7f82080aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 shutdown_connections 2023-10-16T08:19:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.248+0000 7f820c9c4640 1 -- 172.21.15.32:0/3141935035 wait complete. 2023-10-16T08:19:07.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:19:07.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.412+0000 7fd887da1640 1 Processor -- start 2023-10-16T08:19:07.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.412+0000 7fd887da1640 1 -- start start 2023-10-16T08:19:07.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd887da1640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd880164520 0x7fd8800ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd887da1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8800b07f0 0x7fd8800cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd887da1640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8800c96f0 con 0x7fd8800ad960 2023-10-16T08:19:07.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd887da1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd880164900 con 0x7fd8800b07f0 2023-10-16T08:19:07.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd887da1640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800cb950 con 0x7fd880164520 2023-10-16T08:19:07.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8800b07f0 0x7fd8800cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8800b07f0 0x7fd8800cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34648/0 (socket says 172.21.15.32:34648) 2023-10-16T08:19:07.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 -- 172.21.15.32:0/571340633 learned_addr learned my addr 172.21.15.32:0/571340633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:07.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd885b3d640 1 --2- 172.21.15.32:0/571340633 >> v2:172.21.15.186:3300/0 conn(0x7fd880164520 0x7fd8800ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 -- 172.21.15.32:0/571340633 >> v2:172.21.15.186:3300/0 conn(0x7fd880164520 msgr2=0x7fd8800ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:07.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 --2- 172.21.15.32:0/571340633 >> v2:172.21.15.186:3300/0 conn(0x7fd880164520 0x7fd8800ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 -- 172.21.15.32:0/571340633 >> v1:172.21.15.186:6789/0 conn(0x7fd8800ad960 legacy=0x7fd8800b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 -- 172.21.15.32:0/571340633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800cbed0 con 0x7fd8800b07f0 2023-10-16T08:19:07.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.416+0000 7fd88633e640 1 --2- 172.21.15.32:0/571340633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd8800cc870 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd8800cf6f0 tx=0x7fd87801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b093066ac6cb5e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:07.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd884b3b640 1 -- 172.21.15.32:0/571340633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8780a5880 con 0x7fd8800b07f0 2023-10-16T08:19:07.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd884b3b640 1 -- 172.21.15.32:0/571340633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780a59e0 con 0x7fd8800b07f0 2023-10-16T08:19:07.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd884b3b640 1 -- 172.21.15.32:0/571340633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8780a5d70 con 0x7fd8800b07f0 2023-10-16T08:19:07.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- 172.21.15.32:0/571340633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 msgr2=0x7fd8800cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 --2- 172.21.15.32:0/571340633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd8800cc870 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd8800cf6f0 tx=0x7fd87801c9a0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- 172.21.15.32:0/571340633 shutdown_connections 2023-10-16T08:19:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 --2- 172.21.15.32:0/571340633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd8800cc870 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 --2- 172.21.15.32:0/571340633 >> v2:172.21.15.186:3300/0 conn(0x7fd880164520 0x7fd8800ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- 172.21.15.32:0/571340633 >> 172.21.15.32:0/571340633 conn(0x7fd88006c190 msgr2=0x7fd88006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:07.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- 172.21.15.32:0/571340633 shutdown_connections 2023-10-16T08:19:07.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- 172.21.15.32:0/571340633 wait complete. 2023-10-16T08:19:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 Processor -- start 2023-10-16T08:19:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- start start 2023-10-16T08:19:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8800ad960 0x7fd880155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd880155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd880164520 0x7fd88015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880156450 con 0x7fd8800b07f0 2023-10-16T08:19:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880156730 con 0x7fd8800ad960 2023-10-16T08:19:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.420+0000 7fd887da1640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd880156a10 con 0x7fd880164520 2023-10-16T08:19:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd880155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd885b3d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8800ad960 0x7fd880155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd880155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34652/0 (socket says 172.21.15.32:34652) 2023-10-16T08:19:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 -- 172.21.15.32:0/602651209 learned_addr learned my addr 172.21.15.32:0/602651209 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88633e640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd880164520 0x7fd88015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 -- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd880164520 msgr2=0x7fd88015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd880164520 0x7fd88015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 -- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8800ad960 msgr2=0x7fd880155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8800ad960 0x7fd880155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 -- 172.21.15.32:0/602651209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878009b30 con 0x7fd8800b07f0 2023-10-16T08:19:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd88533c640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd880155230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c390 tx=0x7fd87000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:07.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8700052d0 con 0x7fd8800b07f0 2023-10-16T08:19:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd870005430 con 0x7fd8800b07f0 2023-10-16T08:19:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870003820 con 0x7fd8800b07f0 2023-10-16T08:19:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880156cf0 con 0x7fd8800b07f0 2023-10-16T08:19:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880155770 con 0x7fd8800b07f0 2023-10-16T08:19:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880155990 con 0x7fd8800b07f0 2023-10-16T08:19:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd870003980 con 0x7fd8800b07f0 2023-10-16T08:19:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.424+0000 7fd8767fc640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8640664a0 0x7fd864068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.428+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fd870083480 con 0x7fd8800b07f0 2023-10-16T08:19:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.428+0000 7fd885b3d640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8640664a0 0x7fd864068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.428+0000 7fd885b3d640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8640664a0 0x7fd864068960 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0079c0 tx=0x7fd87c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.428+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd870050840 con 0x7fd8800b07f0 2023-10-16T08:19:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.596+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd880074010 con 0x7fd8800b07f0 2023-10-16T08:19:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.596+0000 7fd8767fc640 1 -- 172.21.15.32:0/602651209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd870053e60 con 0x7fd8800b07f0 2023-10-16T08:19:07.600 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:19:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8640664a0 msgr2=0x7fd864068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8640664a0 0x7fd864068960 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0079c0 tx=0x7fd87c008040 comp rx=0 tx=0).stop 2023-10-16T08:19:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 msgr2=0x7fd880155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:07.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd880155230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c390 tx=0x7fd87000c850 comp rx=0 tx=0).stop 2023-10-16T08:19:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 shutdown_connections 2023-10-16T08:19:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd8640664a0 0x7fd864068960 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd880164520 0x7fd88015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8800b07f0 0x7fd880155230 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 --2- 172.21.15.32:0/602651209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8800ad960 0x7fd880155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 >> 172.21.15.32:0/602651209 conn(0x7fd88006c190 msgr2=0x7fd880160960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 shutdown_connections 2023-10-16T08:19:07.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:07.600+0000 7fd887da1640 1 -- 172.21.15.32:0/602651209 wait complete. 2023-10-16T08:19:07.620 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:19:07.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:07.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:07.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51334]: fuse finished with error 0 and tester_r 0 2023-10-16T08:19:07.644 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:13.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:19:13.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:13.747 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:19:13.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:19:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 Processor -- start 2023-10-16T08:19:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 -- start start 2023-10-16T08:19:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f325415ca70 0x7f32540d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32540cf7a0 0x7f32540c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32540c65f0 con 0x7f32540d3550 2023-10-16T08:19:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32540ce820 con 0x7f325415ca70 2023-10-16T08:19:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f325ba17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540cdf80 con 0x7f32540cf7a0 2023-10-16T08:19:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f32597b3640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f325415ca70 0x7f32540d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f3258fb2640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f32540d3550 0x7f32540cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:60308/0 (socket says 172.21.15.32:60308) 2023-10-16T08:19:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f3258fb2640 1 -- 172.21.15.32:0/1969697132 learned_addr learned my addr 172.21.15.32:0/1969697132 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f3259fb4640 1 --2- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32540cf7a0 0x7f32540c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f32597b3640 1 -- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32540cf7a0 msgr2=0x7f32540c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f32597b3640 1 --2- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32540cf7a0 0x7f32540c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f32597b3640 1 -- 172.21.15.32:0/1969697132 >> v1:172.21.15.186:6789/0 conn(0x7f32540d3550 legacy=0x7f32540cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f324bfff640 1 -- 172.21.15.32:0/1969697132 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186525383 0 0) 0x7f32540c65f0 con 0x7f32540d3550 2023-10-16T08:19:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.948+0000 7f32597b3640 1 -- 172.21.15.32:0/1969697132 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540ce260 con 0x7f325415ca70 2023-10-16T08:19:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f32597b3640 1 --2- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f325415ca70 0x7f32540d3010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32500097a0 tx=0x7f325001c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eab8727e3c053bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f324bfff640 1 -- 172.21.15.32:0/1969697132 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500ad640 con 0x7f325415ca70 2023-10-16T08:19:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f324bfff640 1 -- 172.21.15.32:0/1969697132 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32500ad7a0 con 0x7f325415ca70 2023-10-16T08:19:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f324bfff640 1 -- 172.21.15.32:0/1969697132 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500adb30 con 0x7f325415ca70 2023-10-16T08:19:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 -- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f325415ca70 msgr2=0x7f32540d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 --2- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f325415ca70 0x7f32540d3010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32500097a0 tx=0x7f325001c430 comp rx=0 tx=0).stop 2023-10-16T08:19:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 -- 172.21.15.32:0/1969697132 shutdown_connections 2023-10-16T08:19:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 --2- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f32540cf7a0 0x7f32540c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 --2- 172.21.15.32:0/1969697132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f325415ca70 0x7f32540d3010 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 -- 172.21.15.32:0/1969697132 >> 172.21.15.32:0/1969697132 conn(0x7f325409bc50 msgr2=0x7f32540d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:13.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 -- 172.21.15.32:0/1969697132 shutdown_connections 2023-10-16T08:19:13.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.952+0000 7f325ba17640 1 -- 172.21.15.32:0/1969697132 wait complete. 2023-10-16T08:19:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 Processor -- start 2023-10-16T08:19:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 -- start start 2023-10-16T08:19:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32540cf7a0 0x7f325406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:13.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 0x7f3254159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f325415ca70 0x7f325414ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325414f860 con 0x7f32540d3550 2023-10-16T08:19:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325414e2f0 con 0x7f32540cf7a0 2023-10-16T08:19:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3254158540 con 0x7f325415ca70 2023-10-16T08:19:13.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 0x7f3254159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 0x7f3254159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48652/0 (socket says 172.21.15.32:48652) 2023-10-16T08:19:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 -- 172.21.15.32:0/3259581594 learned_addr learned my addr 172.21.15.32:0/3259581594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3259fb4640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f325415ca70 0x7f325414ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f32597b3640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32540cf7a0 0x7f325406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 -- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f325415ca70 msgr2=0x7f325414ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f325415ca70 0x7f325414ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 -- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32540cf7a0 msgr2=0x7f325406bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32540cf7a0 0x7f325406bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 -- 172.21.15.32:0/3259581594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500093b0 con 0x7f32540d3550 2023-10-16T08:19:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3258fb2640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 0x7f3254159ea0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f324400c4a0 tx=0x7f324400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440074e0 con 0x7f32540d3550 2023-10-16T08:19:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244007640 con 0x7f32540d3550 2023-10-16T08:19:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3254157f80 con 0x7f32540d3550 2023-10-16T08:19:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244004a30 con 0x7f32540d3550 2023-10-16T08:19:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.956+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32541505c0 con 0x7f32540d3550 2023-10-16T08:19:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.960+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3254074010 con 0x7f32540d3550 2023-10-16T08:19:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.960+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3244004b90 con 0x7f32540d3550 2023-10-16T08:19:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.960+0000 7f3249ffb640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f32380664a0 0x7f3238068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.960+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f3244083410 con 0x7f32540d3550 2023-10-16T08:19:13.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.960+0000 7f32597b3640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f32380664a0 0x7f3238068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:13.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.960+0000 7f32597b3640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f32380664a0 0x7f3238068960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f32500af000 tx=0x7f32500a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:13.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:13.964+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32440507e0 con 0x7f32540d3550 2023-10-16T08:19:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3254065df0 con 0x7f32540d3550 2023-10-16T08:19:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f3249ffb640 1 -- 172.21.15.32:0/3259581594 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3244053e00 con 0x7f32540d3550 2023-10-16T08:19:14.163 INFO:teuthology.orchestra.run.smithi032.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-16T08:19:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f32380664a0 msgr2=0x7f3238068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f32380664a0 0x7f3238068960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f32500af000 tx=0x7f32500a7070 comp rx=0 tx=0).stop 2023-10-16T08:19:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 msgr2=0x7f3254159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 0x7f3254159ea0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f324400c4a0 tx=0x7f324400c960 comp rx=0 tx=0).stop 2023-10-16T08:19:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 shutdown_connections 2023-10-16T08:19:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f32380664a0 0x7f3238068960 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f325415ca70 0x7f325414ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540d3550 0x7f3254159ea0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 --2- 172.21.15.32:0/3259581594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32540cf7a0 0x7f325406bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 >> 172.21.15.32:0/3259581594 conn(0x7f325409bc50 msgr2=0x7f325406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 shutdown_connections 2023-10-16T08:19:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:14.160+0000 7f325ba17640 1 -- 172.21.15.32:0/3259581594 wait complete. 2023-10-16T08:19:14.183 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:19:14.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:19:14.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-10-16T08:19:14.184 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:19:14.185 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:14.185 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:19:14.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:19:14.186 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:19:14.186 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:19:14.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.187 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-10-16T08:19:14.196 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-10-16T08:19:14.197 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-10-16T08:19:14.198 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-10-16T08:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-10-16T08:19:14.199 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42251sec preferred_lft 42251sec 2023-10-16T08:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-10-16T08:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:14.200 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-10-16T08:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:8d:b1:5a:7c brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.201 INFO:teuthology.orchestra.run.smithi032.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:19:14.202 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:14.203 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:19:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.204 INFO:teuthology.orchestra.run.smithi032.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:19:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-16T08:19:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:19:14.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.206 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:19:14.272 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-10-16T08:19:14.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.275 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-10-16T08:19:14.330 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-10-16T08:19:14.330 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:19:14.331 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:19:14.331 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:19:14.331 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:19:14.331 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:19:14.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.332 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-10-16T08:19:14.332 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-10-16T08:19:14.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:19:14.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:19:14.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:19:14.334 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-10-16T08:19:14.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.426 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:19:14.437 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:19:14.437 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:19:14.437 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:19:14.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-10-16T08:19:14.501 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:19:14.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:19:14.502 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:14.553 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:19:14.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:19:14.554 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:14.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:19:14.670 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:19:14.722 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:19:14.723 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:19:14.723 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:19:14.723 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:19:14.723 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:19:14.723 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:19:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:19:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:19:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:19:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:19:14.724 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:19:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:19:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:19:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:19:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:19:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:19:14.725 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:19:14.726 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:19:14.726 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:19:14.726 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:19:14.726 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:19:14.726 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:19:14.727 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:19:14.727 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:19:14.727 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:19:14.727 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:19:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:19:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:19:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:19:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:14.728 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:14.729 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:14.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.730 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:19:14.785 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:19:14.786 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:19:14.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2023-10-16T08:19:14.858 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:19:14.911 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2023-10-16T08:19:14.911 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2023-10-16T08:19:14.911 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2023-10-16T08:19:14.912 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2023-10-16T08:19:14.912 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2023-10-16T08:19:14.912 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2023-10-16T08:19:14.913 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2023-10-16T08:19:14.913 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2023-10-16T08:19:14.913 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2023-10-16T08:19:14.913 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2023-10-16T08:19:14.914 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2023-10-16T08:19:14.914 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2023-10-16T08:19:14.914 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:19:14.914 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2023-10-16T08:19:14.914 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2023-10-16T08:19:14.915 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2023-10-16T08:19:14.915 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2023-10-16T08:19:14.915 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2023-10-16T08:19:14.916 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2023-10-16T08:19:14.916 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2023-10-16T08:19:14.917 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:19:14.917 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2023-10-16T08:19:14.917 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2023-10-16T08:19:14.917 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2023-10-16T08:19:14.917 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:19:14.918 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:19:14.918 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:19:14.918 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:19:14.918 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2023-10-16T08:19:14.918 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:14.919 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:14.919 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:14.919 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:14.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:14.921 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2023-10-16T08:19:14.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2023-10-16T08:19:14.932+0000 7f3938da4580 -1 init, newargv = 0x55655c705d20 newargc=15 2023-10-16T08:19:14.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51732]: starting ceph client 2023-10-16T08:19:14.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51732]: starting fuse 2023-10-16T08:19:14.944 INFO:teuthology.orchestra.run.smithi032.stdout:43 2023-10-16T08:19:14.945 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-10-16T08:19:14.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:19:14.946 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2023-10-16T08:19:14.946 DEBUG:teuthology.orchestra.run.smithi032:> import re 2023-10-16T08:19:14.946 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:19:14.947 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2023-10-16T08:19:14.947 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:14.947 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2023-10-16T08:19:14.948 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:19:14.948 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2023-10-16T08:19:14.948 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:19:14.949 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:14.949 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2023-10-16T08:19:14.949 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2023-10-16T08:19:14.950 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2023-10-16T08:19:14.950 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2023-10-16T08:19:14.950 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:14.951 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2023-10-16T08:19:14.951 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:19:14.951 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:19:14.952 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:19:14.952 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2023-10-16T08:19:14.953 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2023-10-16T08:19:14.953 DEBUG:teuthology.orchestra.run.smithi032:> return f 2023-10-16T08:19:14.953 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:19:14.954 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:14.954 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2023-10-16T08:19:14.955 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:19:15.071 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.51732.asok 2023-10-16T08:19:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:19:15.082 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51732.asok 2023-10-16T08:19:15.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:15.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51732.asok status 2023-10-16T08:19:15.262 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:15.263 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2023-10-16T08:19:15.263 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T08:19:15.263 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T08:19:15.263 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2023-10-16T08:19:15.263 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2023-10-16T08:19:15.264 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:19:15.264 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "51732", 2023-10-16T08:19:15.264 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2023-10-16T08:19:15.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:15.265 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2023-10-16T08:19:15.265 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2023-10-16T08:19:15.265 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6516, 2023-10-16T08:19:15.265 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2023-10-16T08:19:15.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2023-10-16T08:19:15.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2023-10-16T08:19:15.266 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 6516 2023-10-16T08:19:15.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:15.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:19:15.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:19:15.267 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:19:15.267 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 255796826 2023-10-16T08:19:15.267 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-10-16T08:19:15.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:15.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2023-10-16T08:19:15.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-10-16T08:19:15.268 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:19:15.268 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 255796826 2023-10-16T08:19:15.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-10-16T08:19:15.269 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.6516 192.168.0.1:0/255796826", 2023-10-16T08:19:15.269 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/255796826", 2023-10-16T08:19:15.269 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2023-10-16T08:19:15.269 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 28, 2023-10-16T08:19:15.269 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 54, 2023-10-16T08:19:15.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:19:15.270 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2023-10-16T08:19:15.270 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "testcephfs1" 2023-10-16T08:19:15.270 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:15.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:15.278 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2023-10-16T08:19:15.278 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:15.288 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2023-10-16T08:19:15.288 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:15.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:15.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:15.352 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2023-10-16T08:19:15.353 DEBUG:teuthology.orchestra.run.smithi032:> import os 2023-10-16T08:19:15.353 DEBUG:teuthology.orchestra.run.smithi032:> import errno 2023-10-16T08:19:15.354 DEBUG:teuthology.orchestra.run.smithi032:> 2023-10-16T08:19:15.354 DEBUG:teuthology.orchestra.run.smithi032:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-10-16T08:19:15.354 DEBUG:teuthology.orchestra.run.smithi032:> try: 2023-10-16T08:19:15.355 DEBUG:teuthology.orchestra.run.smithi032:> if True: 2023-10-16T08:19:15.355 DEBUG:teuthology.orchestra.run.smithi032:> ret = os.read(fd, 1024) 2023-10-16T08:19:15.355 DEBUG:teuthology.orchestra.run.smithi032:> else: 2023-10-16T08:19:15.356 DEBUG:teuthology.orchestra.run.smithi032:> os.write(fd, b'"'"'content'"'"') 2023-10-16T08:19:15.356 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2023-10-16T08:19:15.356 DEBUG:teuthology.orchestra.run.smithi032:> if e.errno != errno.EPERM: 2023-10-16T08:19:15.357 DEBUG:teuthology.orchestra.run.smithi032:> raise 2023-10-16T08:19:15.357 DEBUG:teuthology.orchestra.run.smithi032:> else: 2023-10-16T08:19:15.357 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("client does not check permission of data pool") 2023-10-16T08:19:15.358 DEBUG:teuthology.orchestra.run.smithi032:> ' 2023-10-16T08:19:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:Traceback (most recent call last): 2023-10-16T08:19:15.465 INFO:teuthology.orchestra.run.smithi032.stderr: File "", line 15, in 2023-10-16T08:19:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:RuntimeError: client does not check permission of data pool 2023-10-16T08:19:15.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-10-16T08:19:15.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:19:15.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=testcephfs1_data' 2023-10-16T08:19:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 Processor -- start 2023-10-16T08:19:15.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 -- start start 2023-10-16T08:19:15.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5bc0b85d0 0x7fc5bc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fc5bc0d0130 0x7fc5bc0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc0d05a0 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc15e1d0 con 0x7fc5bc0d0130 2023-10-16T08:19:15.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c2ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc161720 con 0x7fc5bc0b89b0 2023-10-16T08:19:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5bc0b85d0 0x7fc5bc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5bc0b85d0 0x7fc5bc15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48668/0 (socket says 172.21.15.32:48668) 2023-10-16T08:19:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 -- 172.21.15.32:0/2262258014 learned_addr learned my addr 172.21.15.32:0/2262258014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c1073640 1 --2- 172.21.15.32:0/2262258014 >> v2:172.21.15.186:3300/0 conn(0x7fc5bc0d0130 0x7fc5bc0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5bbfff640 1 --2- 172.21.15.32:0/2262258014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 -- 172.21.15.32:0/2262258014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 msgr2=0x7fc5bc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 --2- 172.21.15.32:0/2262258014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 -- 172.21.15.32:0/2262258014 >> v2:172.21.15.186:3300/0 conn(0x7fc5bc0d0130 msgr2=0x7fc5bc0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 --2- 172.21.15.32:0/2262258014 >> v2:172.21.15.186:3300/0 conn(0x7fc5bc0d0130 0x7fc5bc0d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.664+0000 7fc5c0872640 1 -- 172.21.15.32:0/2262258014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc160ba0 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c0872640 1 --2- 172.21.15.32:0/2262258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc15dc90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0b4900 tx=0x7fc5b001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a6d4c1bf1b90ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5bb7fe640 1 -- 172.21.15.32:0/2262258014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00ab600 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5bb7fe640 1 -- 172.21.15.32:0/2262258014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b00a7050 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5bb7fe640 1 -- 172.21.15.32:0/2262258014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00ab980 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/2262258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 msgr2=0x7fc5bc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/2262258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc15dc90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0b4900 tx=0x7fc5b001c210 comp rx=0 tx=0).stop 2023-10-16T08:19:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/2262258014 shutdown_connections 2023-10-16T08:19:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/2262258014 >> v2:172.21.15.186:3300/0 conn(0x7fc5bc0d0130 0x7fc5bc0d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/2262258014 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/2262258014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc15dc90 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/2262258014 >> 172.21.15.32:0/2262258014 conn(0x7fc5bc06b750 msgr2=0x7fc5bc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/2262258014 shutdown_connections 2023-10-16T08:19:15.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.668+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/2262258014 wait complete. 2023-10-16T08:19:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 Processor -- start 2023-10-16T08:19:15.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 -- start start 2023-10-16T08:19:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5bc0d0130 0x7fc5bc152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0ca270 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc1516d0 con 0x7fc5bc0d0130 2023-10-16T08:19:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc1519b0 con 0x7fc5bc0b89b0 2023-10-16T08:19:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc0c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48684/0 (socket says 172.21.15.32:48684) 2023-10-16T08:19:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 -- 172.21.15.32:0/526364231 learned_addr learned my addr 172.21.15.32:0/526364231 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c1073640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5bc0d0130 0x7fc5bc152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5bbfff640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 -- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 msgr2=0x7fc5bc0d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 -- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5bc0d0130 msgr2=0x7fc5bc152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5bc0d0130 0x7fc5bc152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 -- 172.21.15.32:0/526364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b00094d0 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c0872640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc0c9d30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00ae040 tx=0x7fc5b001aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00b6420 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b00b65a0 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00b6950 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc151c90 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.672+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc152250 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.676+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc0b5070 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.676+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc5b00b2040 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.676+0000 7fc5b97fa640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc5a00664a0 0x7fc5a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.676+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fc5b011e160 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.676+0000 7fc5bbfff640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc5a00664a0 0x7fc5a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:15.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.676+0000 7fc5bbfff640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc5a00664a0 0x7fc5a0068960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0ca530 tx=0x7fc5a8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.680+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc5b00eb520 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.896+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]} v 0) v1 -- 0x7fc5bc04e170 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.900+0000 7fc5b97fa640 1 -- 172.21.15.32:0/526364231 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=testcephfs1_data"]}]=0 updated caps for client.0 v16) v1 ==== 189+0+146 (secure 0 0 0) 0x7fc5b00eeb40 con 0x7fc5bc0b85d0 2023-10-16T08:19:15.903 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-10-16T08:19:15.903 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAX8SxlOp0aHhAAfDlBSENuky7mnwFxOlt1OQ== 2023-10-16T08:19:15.903 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-10-16T08:19:15.904 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-10-16T08:19:15.904 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw pool=testcephfs1_data" 2023-10-16T08:19:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-10-16T08:19:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.900+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc5a00664a0 msgr2=0x7fc5a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.900+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc5a00664a0 0x7fc5a0068960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0ca530 tx=0x7fc5a8009190 comp rx=0 tx=0).stop 2023-10-16T08:19:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.900+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 msgr2=0x7fc5bc0c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.900+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc0c9d30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00ae040 tx=0x7fc5b001aa50 comp rx=0 tx=0).stop 2023-10-16T08:19:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 shutdown_connections 2023-10-16T08:19:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5bc0d0130 0x7fc5bc152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc5a00664a0 0x7fc5a0068960 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc5bc0b89b0 0x7fc5bc0d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 --2- 172.21.15.32:0/526364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5bc0b85d0 0x7fc5bc0c9d30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 >> 172.21.15.32:0/526364231 conn(0x7fc5bc06b750 msgr2=0x7fc5bc154350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 shutdown_connections 2023-10-16T08:19:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:15.904+0000 7fc5c2ad6640 1 -- 172.21.15.32:0/526364231 wait complete. 2023-10-16T08:19:15.924 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T08:19:15.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:19:16.019 DEBUG:teuthology.orchestra.run.smithi186:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T08:19:16.101 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T08:19:16.102 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:16.102 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:16.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a333e640 1 Processor -- start 2023-10-16T08:19:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a333e640 1 -- start start 2023-10-16T08:19:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a333e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c0b07f0 0x7f099c0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a333e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c0c96f0 con 0x7f099c0ad960 2023-10-16T08:19:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a333e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c0cb950 con 0x7f099c164520 2023-10-16T08:19:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a333e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c164900 con 0x7f099c0b07f0 2023-10-16T08:19:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a08d9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f099c0ad960 0x7f099c0cc870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34826/0 (socket says 172.21.15.32:34826) 2023-10-16T08:19:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a08d9640 1 -- 172.21.15.32:0/1300534704 learned_addr learned my addr 172.21.15.32:0/1300534704 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a18db640 1 --2- 172.21.15.32:0/1300534704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c0b07f0 0x7f099c0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f0993fff640 1 -- 172.21.15.32:0/1300534704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692158513 0 0) 0x7f099c0c96f0 con 0x7f099c0ad960 2023-10-16T08:19:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f0993fff640 1 -- 172.21.15.32:0/1300534704 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0988003530 con 0x7f099c0ad960 2023-10-16T08:19:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a18db640 1 -- 172.21.15.32:0/1300534704 >> v1:172.21.15.186:6789/0 conn(0x7f099c164520 legacy=0x7f099c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f0993fff640 1 -- 172.21.15.32:0/1300534704 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121444031 0 0) 0x7f099c0cb950 con 0x7f099c164520 2023-10-16T08:19:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a18db640 1 -- 172.21.15.32:0/1300534704 >> v1:172.21.15.32:6789/0 conn(0x7f099c0ad960 legacy=0x7f099c0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.296+0000 7f09a18db640 1 -- 172.21.15.32:0/1300534704 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0cbed0 con 0x7f099c0b07f0 2023-10-16T08:19:16.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a18db640 1 --2- 172.21.15.32:0/1300534704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c0b07f0 0x7f099c0ce010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0994009dc0 tx=0x7f099401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7096ca6bb55a637 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f0993fff640 1 -- 172.21.15.32:0/1300534704 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940a5a20 con 0x7f099c0b07f0 2023-10-16T08:19:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f0993fff640 1 -- 172.21.15.32:0/1300534704 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09940a5b80 con 0x7f099c0b07f0 2023-10-16T08:19:16.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f0993fff640 1 -- 172.21.15.32:0/1300534704 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940ad440 con 0x7f099c0b07f0 2023-10-16T08:19:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 -- 172.21.15.32:0/1300534704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c0b07f0 msgr2=0x7f099c0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 --2- 172.21.15.32:0/1300534704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c0b07f0 0x7f099c0ce010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0994009dc0 tx=0x7f099401c940 comp rx=0 tx=0).stop 2023-10-16T08:19:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 -- 172.21.15.32:0/1300534704 shutdown_connections 2023-10-16T08:19:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 --2- 172.21.15.32:0/1300534704 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c0b07f0 0x7f099c0ce010 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 -- 172.21.15.32:0/1300534704 >> 172.21.15.32:0/1300534704 conn(0x7f099c06c190 msgr2=0x7f099c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 -- 172.21.15.32:0/1300534704 shutdown_connections 2023-10-16T08:19:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.300+0000 7f09a333e640 1 -- 172.21.15.32:0/1300534704 wait complete. 2023-10-16T08:19:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 Processor -- start 2023-10-16T08:19:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 -- start start 2023-10-16T08:19:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f099c0ad960 0x7f099c162f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 0x7f099c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c164520 0x7f099c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c161ef0 con 0x7f099c0b07f0 2023-10-16T08:19:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1621d0 con 0x7f099c0ad960 2023-10-16T08:19:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1624b0 con 0x7f099c164520 2023-10-16T08:19:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 0x7f099c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 0x7f099c15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48694/0 (socket says 172.21.15.32:48694) 2023-10-16T08:19:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 -- 172.21.15.32:0/1519693307 learned_addr learned my addr 172.21.15.32:0/1519693307 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a10da640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f099c0ad960 0x7f099c162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a18db640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c164520 0x7f099c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 -- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c164520 msgr2=0x7f099c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c164520 0x7f099c153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 -- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f099c0ad960 msgr2=0x7f099c162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f099c0ad960 0x7f099c162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 -- 172.21.15.32:0/1519693307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009a70 con 0x7f099c0b07f0 2023-10-16T08:19:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a10da640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f099c0ad960 0x7f099c162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a08d9640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 0x7f099c15b910 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c390 tx=0x7f098c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c005cf0 con 0x7f099c0b07f0 2023-10-16T08:19:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c005e70 con 0x7f099c0b07f0 2023-10-16T08:19:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0072f0 con 0x7f099c0b07f0 2023-10-16T08:19:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c162790 con 0x7f099c0b07f0 2023-10-16T08:19:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.304+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c15be50 con 0x7f099c0b07f0 2023-10-16T08:19:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.308+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c0b0f60 con 0x7f099c0b07f0 2023-10-16T08:19:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.308+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f098c007450 con 0x7f099c0b07f0 2023-10-16T08:19:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.308+0000 7f0991ffb640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f09800664a0 0x7f0980068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.308+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f098c083570 con 0x7f099c0b07f0 2023-10-16T08:19:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.308+0000 7f09a10da640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f09800664a0 0x7f0980068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.308+0000 7f09a10da640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f09800664a0 0x7f0980068960 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0998000f70 tx=0x7f0998009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.312+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f098c0509f0 con 0x7f099c0b07f0 2023-10-16T08:19:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f099c074010 con 0x7f099c0b07f0 2023-10-16T08:19:16.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f0991ffb640 1 -- 172.21.15.32:0/1519693307 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f098c054010 con 0x7f099c0b07f0 2023-10-16T08:19:16.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:16.475 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f09800664a0 msgr2=0x7f0980068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f09800664a0 0x7f0980068960 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0998000f70 tx=0x7f0998009040 comp rx=0 tx=0).stop 2023-10-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 msgr2=0x7f099c15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 0x7f099c15b910 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c390 tx=0x7f098c00c850 comp rx=0 tx=0).stop 2023-10-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 shutdown_connections 2023-10-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f09800664a0 0x7f0980068960 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f099c164520 0x7f099c153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f099c0b07f0 0x7f099c15b910 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 --2- 172.21.15.32:0/1519693307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f099c0ad960 0x7f099c162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 >> 172.21.15.32:0/1519693307 conn(0x7f099c06c190 msgr2=0x7f099c0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 shutdown_connections 2023-10-16T08:19:16.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.472+0000 7f09a333e640 1 -- 172.21.15.32:0/1519693307 wait complete. 2023-10-16T08:19:16.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:18:48.142496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:18:48.637534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:19:16.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:19:16.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 Processor -- start 2023-10-16T08:19:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 -- start start 2023-10-16T08:19:16.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7ff41815cba0 0x7ff4180d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 0x7ff4180b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4180d1670 con 0x7ff4180b84e0 2023-10-16T08:19:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4180d1460 con 0x7ff41815cba0 2023-10-16T08:19:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41fcc9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180d1950 con 0x7ff4180d0210 2023-10-16T08:19:16.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41d264640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4180b84e0 0x7ff4180cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34828/0 (socket says 172.21.15.32:34828) 2023-10-16T08:19:16.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41d264640 1 -- 172.21.15.32:0/1446494081 learned_addr learned my addr 172.21.15.32:0/1446494081 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41e266640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 0x7ff4180b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41da65640 1 --2- 172.21.15.32:0/1446494081 >> v2:172.21.15.186:3300/0 conn(0x7ff41815cba0 0x7ff4180d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808771958 0 0) 0x7ff4180d1670 con 0x7ff4180b84e0 2023-10-16T08:19:16.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff404003530 con 0x7ff4180b84e0 2023-10-16T08:19:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.640+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175116358 0 0) 0x7ff404003530 con 0x7ff4180b84e0 2023-10-16T08:19:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 msgr2=0x7ff4180b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 0x7ff4180b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 >> v2:172.21.15.186:3300/0 conn(0x7ff41815cba0 msgr2=0x7ff4180d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 --2- 172.21.15.32:0/1446494081 >> v2:172.21.15.186:3300/0 conn(0x7ff41815cba0 0x7ff4180d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180d1c30 con 0x7ff4180b84e0 2023-10-16T08:19:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7ff408001680 con 0x7ff4180b84e0 2023-10-16T08:19:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 >> v1:172.21.15.32:6789/0 conn(0x7ff4180b84e0 legacy=0x7ff4180cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 0x7ff404003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4040047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff404004d10 0x7ff404005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180d1460 con 0x7ff404003f00 2023-10-16T08:19:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180d1950 con 0x7ff404004d10 2023-10-16T08:19:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40400b990 con 0x7ff4180d0210 2023-10-16T08:19:16.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4040047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41da65640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 0x7ff404003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41e266640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff404004d10 0x7ff404005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 -- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 msgr2=0x7ff404003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180d0210 0x7ff404003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 -- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff404004d10 msgr2=0x7ff404005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff404004d10 0x7ff404005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 -- 172.21.15.32:0/1446494081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404003530 con 0x7ff404003f00 2023-10-16T08:19:16.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41e266640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff404004d10 0x7ff404005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41d264640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4040047d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff40800c240 tx=0x7ff40801f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad808676ec220da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408005ae0 con 0x7ff404003f00 2023-10-16T08:19:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4080ab070 con 0x7ff404003f00 2023-10-16T08:19:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.644+0000 7ff41ca63640 1 -- 172.21.15.32:0/1446494081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408007070 con 0x7ff404003f00 2023-10-16T08:19:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1446494081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 msgr2=0x7ff4040047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1446494081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4040047d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff40800c240 tx=0x7ff40801f370 comp rx=0 tx=0).stop 2023-10-16T08:19:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41e266640 1 -- 172.21.15.32:0/1446494081 reap_dead start 2023-10-16T08:19:16.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1446494081 shutdown_connections 2023-10-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1446494081 >> 172.21.15.32:0/1446494081 conn(0x7ff4180719c0 msgr2=0x7ff418071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1446494081 shutdown_connections 2023-10-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1446494081 wait complete. 2023-10-16T08:19:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 Processor -- start 2023-10-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- start start 2023-10-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4180c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180c7100 0x7ff418153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4180b71e0 0x7ff4180b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418154570 con 0x7ff404003f00 2023-10-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181f1f20 con 0x7ff4180b71e0 2023-10-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41fcc9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180b7ba0 con 0x7ff4180c7100 2023-10-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41da65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4180c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41da65640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4180c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48712/0 (socket says 172.21.15.32:48712) 2023-10-16T08:19:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41da65640 1 -- 172.21.15.32:0/1274706697 learned_addr learned my addr 172.21.15.32:0/1274706697 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41e266640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4180b71e0 0x7ff4180b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.648+0000 7ff41d264640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180c7100 0x7ff418153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41da65640 1 -- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180c7100 msgr2=0x7ff418153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41da65640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180c7100 0x7ff418153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41da65640 1 -- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4180b71e0 msgr2=0x7ff4180b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41da65640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4180b71e0 0x7ff4180b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41da65640 1 -- 172.21.15.32:0/1274706697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180c8810 con 0x7ff404003f00 2023-10-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41d264640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180c7100 0x7ff418153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41da65640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4180c8150 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff414011ad0 tx=0x7ff414011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41400fbb0 con 0x7ff404003f00 2023-10-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41400fd10 con 0x7ff404003f00 2023-10-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414014680 con 0x7ff404003f00 2023-10-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff418153bd0 con 0x7ff404003f00 2023-10-16T08:19:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff418154190 con 0x7ff404003f00 2023-10-16T08:19:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41804e170 con 0x7ff404003f00 2023-10-16T08:19:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4140147e0 con 0x7ff404003f00 2023-10-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff40e7fc640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3fc0664d0 0x7ff3fc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7ff4140848d0 con 0x7ff404003f00 2023-10-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.652+0000 7ff41d264640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3fc0664d0 0x7ff3fc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.656+0000 7ff41d264640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3fc0664d0 0x7ff3fc068990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff4080ae000 tx=0x7ff408001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.656+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff414051b50 con 0x7ff404003f00 2023-10-16T08:19:16.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff41815a790 con 0x7ff404003f00 2023-10-16T08:19:16.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff40e7fc640 1 -- 172.21.15.32:0/1274706697 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff414022020 con 0x7ff404003f00 2023-10-16T08:19:16.819 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:19:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3fc0664d0 msgr2=0x7ff3fc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3fc0664d0 0x7ff3fc068990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff4080ae000 tx=0x7ff408001a80 comp rx=0 tx=0).stop 2023-10-16T08:19:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 msgr2=0x7ff4180c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4180c8150 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff414011ad0 tx=0x7ff414011f90 comp rx=0 tx=0).stop 2023-10-16T08:19:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 shutdown_connections 2023-10-16T08:19:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4180b71e0 0x7ff4180b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7ff3fc0664d0 0x7ff3fc068990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7ff4180c7100 0x7ff418153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 --2- 172.21.15.32:0/1274706697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff404003f00 0x7ff4180c8150 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 >> 172.21.15.32:0/1274706697 conn(0x7ff4180719c0 msgr2=0x7ff418159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 shutdown_connections 2023-10-16T08:19:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.816+0000 7ff41fcc9640 1 -- 172.21.15.32:0/1274706697 wait complete. 2023-10-16T08:19:16.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:19:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 Processor -- start 2023-10-16T08:19:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 -- start start 2023-10-16T08:19:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f579815cba0 0x7f57980d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57980b84e0 0x7f57980cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57980d1670 con 0x7f57980d0040 2023-10-16T08:19:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57980d1460 con 0x7f57980b84e0 2023-10-16T08:19:16.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f579db2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980d04b0 con 0x7f579815cba0 2023-10-16T08:19:16.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5796ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57980b84e0 0x7f57980cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5796ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57980b84e0 0x7f57980cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48716/0 (socket says 172.21.15.32:48716) 2023-10-16T08:19:16.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5796ffd640 1 -- 172.21.15.32:0/3741489924 learned_addr learned my addr 172.21.15.32:0/3741489924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:16.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5797fff640 1 --1- 172.21.15.32:0/3741489924 >> v1:172.21.15.32:6789/0 conn(0x7f57980d0040 0x7f57980d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34842/0 (socket says 172.21.15.32:34842) 2023-10-16T08:19:16.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f57977fe640 1 --2- 172.21.15.32:0/3741489924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f579815cba0 0x7f57980d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:16.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5796ffd640 1 -- 172.21.15.32:0/3741489924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f579815cba0 msgr2=0x7f57980d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5796ffd640 1 --2- 172.21.15.32:0/3741489924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f579815cba0 0x7f57980d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.984+0000 7f5796ffd640 1 -- 172.21.15.32:0/3741489924 >> v1:172.21.15.32:6789/0 conn(0x7f57980d0040 legacy=0x7f57980d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f57967fc640 1 -- 172.21.15.32:0/3741489924 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592143219 0 0) 0x7f57980d1670 con 0x7f57980d0040 2023-10-16T08:19:16.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f5796ffd640 1 -- 172.21.15.32:0/3741489924 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980d1950 con 0x7f57980b84e0 2023-10-16T08:19:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f5796ffd640 1 --2- 172.21.15.32:0/3741489924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57980b84e0 0x7f57980cfb00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed1796c39b32452 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f57967fc640 1 -- 172.21.15.32:0/3741489924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0ab660 con 0x7f57980b84e0 2023-10-16T08:19:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f57967fc640 1 -- 172.21.15.32:0/3741489924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c0ab7c0 con 0x7f57980b84e0 2023-10-16T08:19:16.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f57967fc640 1 -- 172.21.15.32:0/3741489924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0abb50 con 0x7f57980b84e0 2023-10-16T08:19:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 -- 172.21.15.32:0/3741489924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57980b84e0 msgr2=0x7f57980cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 --2- 172.21.15.32:0/3741489924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57980b84e0 0x7f57980cfb00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c01c540 comp rx=0 tx=0).stop 2023-10-16T08:19:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 -- 172.21.15.32:0/3741489924 shutdown_connections 2023-10-16T08:19:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 --2- 172.21.15.32:0/3741489924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57980b84e0 0x7f57980cfb00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 --2- 172.21.15.32:0/3741489924 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f579815cba0 0x7f57980d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 -- 172.21.15.32:0/3741489924 >> 172.21.15.32:0/3741489924 conn(0x7f57980719c0 msgr2=0x7f5798071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 -- 172.21.15.32:0/3741489924 shutdown_connections 2023-10-16T08:19:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.988+0000 7f579db2a640 1 -- 172.21.15.32:0/3741489924 wait complete. 2023-10-16T08:19:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 Processor -- start 2023-10-16T08:19:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 -- start start 2023-10-16T08:19:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57980b84e0 0x7f5798149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57980d0040 0x7f57980c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 0x7f57980c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c6b10 con 0x7f579815cba0 2023-10-16T08:19:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57981506f0 con 0x7f57980b84e0 2023-10-16T08:19:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5798155900 con 0x7f57980d0040 2023-10-16T08:19:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 0x7f57980c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 0x7f57980c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48720/0 (socket says 172.21.15.32:48720) 2023-10-16T08:19:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 -- 172.21.15.32:0/2509527778 learned_addr learned my addr 172.21.15.32:0/2509527778 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5796ffd640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57980d0040 0x7f57980c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f57977fe640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57980b84e0 0x7f5798149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 -- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57980d0040 msgr2=0x7f57980c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57980d0040 0x7f57980c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 -- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57980b84e0 msgr2=0x7f5798149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57980b84e0 0x7f5798149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 -- 172.21.15.32:0/2509527778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0094d0 con 0x7f579815cba0 2023-10-16T08:19:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f5797fff640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 0x7f57980c8410 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f579000c810 tx=0x7f579000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000a640 con 0x7f579815cba0 2023-10-16T08:19:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000a7c0 con 0x7f579815cba0 2023-10-16T08:19:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000ab70 con 0x7f579815cba0 2023-10-16T08:19:17.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.992+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57980c8950 con 0x7f579815cba0 2023-10-16T08:19:17.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.996+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5798155b50 con 0x7f579815cba0 2023-10-16T08:19:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.996+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57980672e0 con 0x7f579815cba0 2023-10-16T08:19:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.996+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f579000acd0 con 0x7f579815cba0 2023-10-16T08:19:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.996+0000 7f577ffff640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57840664d0 0x7f5784068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.996+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f57900833a0 con 0x7f579815cba0 2023-10-16T08:19:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:16.996+0000 7f57977fe640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57840664d0 0x7f5784068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.000+0000 7f57977fe640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57840664d0 0x7f5784068990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57880079c0 tx=0x7f5788008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.000+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5790050760 con 0x7f579815cba0 2023-10-16T08:19:17.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7f579804e170 con 0x7f579815cba0 2023-10-16T08:19:17.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f577ffff640 1 -- 172.21.15.32:0/2509527778 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v54) v1 ==== 109+0+11 (secure 0 0 0) 0x7f5790018020 con 0x7f579815cba0 2023-10-16T08:19:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:19:17.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57840664d0 msgr2=0x7f5784068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57840664d0 0x7f5784068990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57880079c0 tx=0x7f5788008040 comp rx=0 tx=0).stop 2023-10-16T08:19:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 msgr2=0x7f57980c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 0x7f57980c8410 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f579000c810 tx=0x7f579000ccd0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 shutdown_connections 2023-10-16T08:19:17.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579815cba0 0x7f57980c8410 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f57840664d0 0x7f5784068990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f57980d0040 0x7f57980c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 --2- 172.21.15.32:0/2509527778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57980b84e0 0x7f5798149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 >> 172.21.15.32:0/2509527778 conn(0x7f57980719c0 msgr2=0x7f57980b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:17.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 shutdown_connections 2023-10-16T08:19:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.160+0000 7f579db2a640 1 -- 172.21.15.32:0/2509527778 wait complete. 2023-10-16T08:19:17.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:19:17.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 Processor -- start 2023-10-16T08:19:17.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 -- start start 2023-10-16T08:19:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1e8c06d010 0x7f1e8c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c0c26b0 0x7f1e8c0c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c15ae30 con 0x7f1e8c069e90 2023-10-16T08:19:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8c0cb090 con 0x7f1e8c06d010 2023-10-16T08:19:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e93009640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c06d470 con 0x7f1e8c0c26b0 2023-10-16T08:19:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e8bfff640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f1e8c06d010 0x7f1e8c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e90da5640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f1e8c069e90 0x7f1e8c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:60320/0 (socket says 172.21.15.32:60320) 2023-10-16T08:19:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e915a6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c0c26b0 0x7f1e8c0c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e90da5640 1 -- 172.21.15.32:0/466786939 learned_addr learned my addr 172.21.15.32:0/466786939 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e8bfff640 1 -- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c0c26b0 msgr2=0x7f1e8c0c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c0c26b0 0x7f1e8c0c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e8bfff640 1 -- 172.21.15.32:0/466786939 >> v1:172.21.15.186:6789/0 conn(0x7f1e8c069e90 legacy=0x7f1e8c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.332+0000 7f1e8bfff640 1 -- 172.21.15.32:0/466786939 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0ca510 con 0x7f1e8c06d010 2023-10-16T08:19:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c06d010 0x7f1e8c0c9ce0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e780097c0 tx=0x7f1e7801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=675eb91fda1e53e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/466786939 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780a8970 con 0x7f1e8c06d010 2023-10-16T08:19:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/466786939 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e780a8ad0 con 0x7f1e8c06d010 2023-10-16T08:19:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e8b7fe640 1 -- 172.21.15.32:0/466786939 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780a8e60 con 0x7f1e8c06d010 2023-10-16T08:19:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 -- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c06d010 msgr2=0x7f1e8c0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 --2- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c06d010 0x7f1e8c0c9ce0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e780097c0 tx=0x7f1e7801ec70 comp rx=0 tx=0).stop 2023-10-16T08:19:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 -- 172.21.15.32:0/466786939 shutdown_connections 2023-10-16T08:19:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 --2- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c0c26b0 0x7f1e8c0c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 --2- 172.21.15.32:0/466786939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c06d010 0x7f1e8c0c9ce0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 -- 172.21.15.32:0/466786939 >> 172.21.15.32:0/466786939 conn(0x7f1e8c0b0da0 msgr2=0x7f1e8c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 -- 172.21.15.32:0/466786939 shutdown_connections 2023-10-16T08:19:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.336+0000 7f1e93009640 1 -- 172.21.15.32:0/466786939 wait complete. 2023-10-16T08:19:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 Processor -- start 2023-10-16T08:19:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 -- start start 2023-10-16T08:19:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c069e90 0x7f1e8c153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 0x7f1e8c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c0c26b0 0x7f1e8c15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0c2bd0 con 0x7f1e8c06d010 2023-10-16T08:19:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c15c350 con 0x7f1e8c0c26b0 2023-10-16T08:19:17.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c15c630 con 0x7f1e8c069e90 2023-10-16T08:19:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 0x7f1e8c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 0x7f1e8c15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48724/0 (socket says 172.21.15.32:48724) 2023-10-16T08:19:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 -- 172.21.15.32:0/3616197106 learned_addr learned my addr 172.21.15.32:0/3616197106 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e90da5640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c069e90 0x7f1e8c153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e915a6640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c0c26b0 0x7f1e8c15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 -- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c069e90 msgr2=0x7f1e8c153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c069e90 0x7f1e8c153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 -- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c0c26b0 msgr2=0x7f1e8c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c0c26b0 0x7f1e8c15bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 -- 172.21.15.32:0/3616197106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78009420 con 0x7f1e8c06d010 2023-10-16T08:19:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e8bfff640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 0x7f1e8c15b5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b3040 tx=0x7f1e7801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780b9630 con 0x7f1e8c06d010 2023-10-16T08:19:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e780b9790 con 0x7f1e8c06d010 2023-10-16T08:19:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780b9b40 con 0x7f1e8c06d010 2023-10-16T08:19:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c15c910 con 0x7f1e8c06d010 2023-10-16T08:19:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.340+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c154010 con 0x7f1e8c06d010 2023-10-16T08:19:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.344+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c1542c0 con 0x7f1e8c06d010 2023-10-16T08:19:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.344+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1e780b7040 con 0x7f1e8c06d010 2023-10-16T08:19:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.344+0000 7f1e897fa640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1e700664a0 0x7f1e70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.344+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f1e78121e30 con 0x7f1e8c06d010 2023-10-16T08:19:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.344+0000 7f1e90da5640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1e700664a0 0x7f1e70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.344+0000 7f1e90da5640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1e700664a0 0x7f1e70068960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1e80005890 tx=0x7f1e80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.348+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1e780ef160 con 0x7f1e8c06d010 2023-10-16T08:19:17.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:19:17.398 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:19:17.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.504+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f1e8c04e170 con 0x7f1e8c06d010 2023-10-16T08:19:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.504+0000 7f1e897fa640 1 -- 172.21.15.32:0/3616197106 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v54) v1 ==== 105+0+11 (secure 0 0 0) 0x7f1e780b5020 con 0x7f1e8c06d010 2023-10-16T08:19:17.509 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:19:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1e700664a0 msgr2=0x7f1e70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1e700664a0 0x7f1e70068960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1e80005890 tx=0x7f1e80009040 comp rx=0 tx=0).stop 2023-10-16T08:19:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 msgr2=0x7f1e8c15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 0x7f1e8c15b5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b3040 tx=0x7f1e7801ba10 comp rx=0 tx=0).stop 2023-10-16T08:19:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 shutdown_connections 2023-10-16T08:19:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8c0c26b0 0x7f1e8c15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e8c06d010 0x7f1e8c15b5d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f1e700664a0 0x7f1e70068960 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 --2- 172.21.15.32:0/3616197106 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f1e8c069e90 0x7f1e8c153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 >> 172.21.15.32:0/3616197106 conn(0x7f1e8c0b0da0 msgr2=0x7f1e8c0ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:17.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 shutdown_connections 2023-10-16T08:19:17.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.508+0000 7f1e93009640 1 -- 172.21.15.32:0/3616197106 wait complete. 2023-10-16T08:19:17.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:19:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 Processor -- start 2023-10-16T08:19:17.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 -- start start 2023-10-16T08:19:17.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f93601597c0 0x7f93600ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93600cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93600723d0 con 0x7f93600adc10 2023-10-16T08:19:17.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600717c0 con 0x7f93601597c0 2023-10-16T08:19:17.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936869e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9360071aa0 con 0x7f9360159ba0 2023-10-16T08:19:17.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365c39640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93600cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936643a640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f93601597c0 0x7f93600ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9366c3b640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f93600adc10 0x7f93600cb350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:60330/0 (socket says 172.21.15.32:60330) 2023-10-16T08:19:17.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9366c3b640 1 -- 172.21.15.32:0/2275232129 learned_addr learned my addr 172.21.15.32:0/2275232129 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:17.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365438640 1 -- 172.21.15.32:0/2275232129 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804720129 0 0) 0x7f93600723d0 con 0x7f93600adc10 2023-10-16T08:19:17.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365438640 1 -- 172.21.15.32:0/2275232129 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f934c003530 con 0x7f93600adc10 2023-10-16T08:19:17.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365c39640 1 -- 172.21.15.32:0/2275232129 >> v2:172.21.15.186:3300/0 conn(0x7f93601597c0 msgr2=0x7f93600ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:17.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365c39640 1 --2- 172.21.15.32:0/2275232129 >> v2:172.21.15.186:3300/0 conn(0x7f93601597c0 0x7f93600ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365c39640 1 -- 172.21.15.32:0/2275232129 >> v1:172.21.15.186:6789/0 conn(0x7f93600adc10 legacy=0x7f93600cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f9365c39640 1 -- 172.21.15.32:0/2275232129 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360071d80 con 0x7f9360159ba0 2023-10-16T08:19:17.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.672+0000 7f936643a640 1 --2- 172.21.15.32:0/2275232129 >> v2:172.21.15.186:3300/0 conn(0x7f93601597c0 0x7f93600ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:17.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f9365c39640 1 --2- 172.21.15.32:0/2275232129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93600cae10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f93500097c0 tx=0x7f935001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80afeed93a166545 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f9365438640 1 -- 172.21.15.32:0/2275232129 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500ab620 con 0x7f9360159ba0 2023-10-16T08:19:17.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f9365438640 1 -- 172.21.15.32:0/2275232129 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500ab780 con 0x7f9360159ba0 2023-10-16T08:19:17.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f9365438640 1 -- 172.21.15.32:0/2275232129 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500abb10 con 0x7f9360159ba0 2023-10-16T08:19:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 -- 172.21.15.32:0/2275232129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 msgr2=0x7f93600cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 --2- 172.21.15.32:0/2275232129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93600cae10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f93500097c0 tx=0x7f935001c480 comp rx=0 tx=0).stop 2023-10-16T08:19:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 -- 172.21.15.32:0/2275232129 shutdown_connections 2023-10-16T08:19:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 --2- 172.21.15.32:0/2275232129 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93600cae10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 --2- 172.21.15.32:0/2275232129 >> v2:172.21.15.186:3300/0 conn(0x7f93601597c0 0x7f93600ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 -- 172.21.15.32:0/2275232129 >> 172.21.15.32:0/2275232129 conn(0x7f936006c190 msgr2=0x7f936006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 -- 172.21.15.32:0/2275232129 shutdown_connections 2023-10-16T08:19:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.676+0000 7f936869e640 1 -- 172.21.15.32:0/2275232129 wait complete. 2023-10-16T08:19:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 Processor -- start 2023-10-16T08:19:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 -- start start 2023-10-16T08:19:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 0x7f936015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93601597c0 0x7f93600c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93601f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601f8dc0 con 0x7f93600adc10 2023-10-16T08:19:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600c85b0 con 0x7f93601597c0 2023-10-16T08:19:17.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936014b370 con 0x7f9360159ba0 2023-10-16T08:19:17.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 0x7f936015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 0x7f936015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48736/0 (socket says 172.21.15.32:48736) 2023-10-16T08:19:17.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 -- 172.21.15.32:0/838218198 learned_addr learned my addr 172.21.15.32:0/838218198 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:17.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f9365c39640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93601597c0 0x7f93600c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f9366c3b640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93601f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 -- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 msgr2=0x7f93601f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93601f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 -- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93601597c0 msgr2=0x7f93600c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93601597c0 0x7f93600c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 -- 172.21.15.32:0/838218198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c009420 con 0x7f93600adc10 2023-10-16T08:19:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f9366c3b640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93601f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936643a640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 0x7f936015c710 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f935c00ec10 tx=0x7f935c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00cc30 con 0x7f93600adc10 2023-10-16T08:19:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00cd90 con 0x7f93600adc10 2023-10-16T08:19:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9350009420 con 0x7f93600adc10 2023-10-16T08:19:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c013690 con 0x7f93600adc10 2023-10-16T08:19:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.680+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601f9720 con 0x7f93600adc10 2023-10-16T08:19:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.684+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93601f9030 con 0x7f93600adc10 2023-10-16T08:19:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.684+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f935c0137f0 con 0x7f93600adc10 2023-10-16T08:19:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.684+0000 7f9356ffd640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f93440664d0 0x7f9344068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.684+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f935c084060 con 0x7f93600adc10 2023-10-16T08:19:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.684+0000 7f9365c39640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f93440664d0 0x7f9344068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.684+0000 7f9365c39640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f93440664d0 0x7f9344068990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f93500af000 tx=0x7f93500a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:17.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.688+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f935c0b5090 con 0x7f93600adc10 2023-10-16T08:19:17.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.860+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93601f9720 con 0x7f93600adc10 2023-10-16T08:19:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.860+0000 7f9356ffd640 1 -- 172.21.15.32:0/838218198 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f93601f9720 con 0x7f93600adc10 2023-10-16T08:19:17.864 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-10-16T08:19:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f93440664d0 msgr2=0x7f9344068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f93440664d0 0x7f9344068990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f93500af000 tx=0x7f93500a7000 comp rx=0 tx=0).stop 2023-10-16T08:19:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 msgr2=0x7f936015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 0x7f936015c710 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f935c00ec10 tx=0x7f935c00c520 comp rx=0 tx=0).stop 2023-10-16T08:19:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 shutdown_connections 2023-10-16T08:19:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f93440664d0 0x7f9344068990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9360159ba0 0x7f93601f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93601597c0 0x7f93600c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 --2- 172.21.15.32:0/838218198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93600adc10 0x7f936015c710 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:17.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 >> 172.21.15.32:0/838218198 conn(0x7f936006c190 msgr2=0x7f93600c9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:17.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 shutdown_connections 2023-10-16T08:19:17.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:17.864+0000 7f936869e640 1 -- 172.21.15.32:0/838218198 wait complete. 2023-10-16T08:19:17.885 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2023-10-16T08:19:17.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:17.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:19:17.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51732]: fuse finished with error 0 and tester_r 0 2023-10-16T08:19:17.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:19:17.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:17.909 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:19:18.074 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:19:18.075 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:18.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:19:18.085 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:19:18.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:18.086 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T08:19:18.139 INFO:teuthology.orchestra.run.smithi186.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-16T08:19:18.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:19:18.140 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:18.141 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:18.151 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:19:18.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:19:18.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:18.152 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T08:19:18.200 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-16T08:19:18.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:19:18.202 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:19:18.202 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:18.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:19:18.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:19:18.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:18.213 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T08:19:18.263 INFO:teuthology.orchestra.run.smithi186.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-16T08:19:18.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:19:18.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:19:18.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.404+0000 7f5217d3e640 1 Processor -- start 2023-10-16T08:19:18.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.404+0000 7f5217d3e640 1 -- start start 2023-10-16T08:19:18.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5217d3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521015a800 0x7f52100cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5217d3e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f52100b2600 0x7f52100704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5217d3e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52100c7110 con 0x7f52100b21c0 2023-10-16T08:19:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5217d3e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100cf820 con 0x7f52100b2600 2023-10-16T08:19:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5217d3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100ce780 con 0x7f521015a800 2023-10-16T08:19:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52162db640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f52100b2600 0x7f52100704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5215ada640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521015a800 0x7f52100cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52152d9640 1 --1- >> v1:172.21.15.186:6789/0 conn(0x7f52100b21c0 0x7f521006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:60344/0 (socket says 172.21.15.32:60344) 2023-10-16T08:19:18.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52152d9640 1 -- 172.21.15.32:0/2333925216 learned_addr learned my addr 172.21.15.32:0/2333925216 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:18.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5214ad8640 1 -- 172.21.15.32:0/2333925216 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341968476 0 0) 0x7f52100c7110 con 0x7f52100b21c0 2023-10-16T08:19:18.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f5214ad8640 1 -- 172.21.15.32:0/2333925216 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51fc003530 con 0x7f52100b21c0 2023-10-16T08:19:18.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52162db640 1 -- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521015a800 msgr2=0x7f52100cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52162db640 1 --2- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521015a800 0x7f52100cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52162db640 1 -- 172.21.15.32:0/2333925216 >> v1:172.21.15.186:6789/0 conn(0x7f52100b21c0 legacy=0x7f521006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.408+0000 7f52162db640 1 -- 172.21.15.32:0/2333925216 --> v2:172.21.15.186:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100cea60 con 0x7f52100b2600 2023-10-16T08:19:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f52162db640 1 --2- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52100b2600 0x7f52100704b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5208009d90 tx=0x7f520801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24552bf0161b2bf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5214ad8640 1 -- 172.21.15.32:0/2333925216 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52080a59e0 con 0x7f52100b2600 2023-10-16T08:19:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5214ad8640 1 -- 172.21.15.32:0/2333925216 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52080a5b40 con 0x7f52100b2600 2023-10-16T08:19:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5214ad8640 1 -- 172.21.15.32:0/2333925216 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52080ad440 con 0x7f52100b2600 2023-10-16T08:19:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 -- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52100b2600 msgr2=0x7f52100704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 --2- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52100b2600 0x7f52100704b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5208009d90 tx=0x7f520801c920 comp rx=0 tx=0).stop 2023-10-16T08:19:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 -- 172.21.15.32:0/2333925216 shutdown_connections 2023-10-16T08:19:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 --2- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52100b2600 0x7f52100704b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 --2- 172.21.15.32:0/2333925216 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f521015a800 0x7f52100cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 -- 172.21.15.32:0/2333925216 >> 172.21.15.32:0/2333925216 conn(0x7f52100cd1c0 msgr2=0x7f52100cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:18.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 -- 172.21.15.32:0/2333925216 shutdown_connections 2023-10-16T08:19:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 -- 172.21.15.32:0/2333925216 wait complete. 2023-10-16T08:19:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 Processor -- start 2023-10-16T08:19:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.412+0000 7f5217d3e640 1 -- start start 2023-10-16T08:19:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 0x7f52101519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52100b2600 0x7f52100ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521015a800 0x7f5210159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b3c50 con 0x7f52100b21c0 2023-10-16T08:19:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b3340 con 0x7f521015a800 2023-10-16T08:19:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b4070 con 0x7f52100b2600 2023-10-16T08:19:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 0x7f52101519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 0x7f52101519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48740/0 (socket says 172.21.15.32:48740) 2023-10-16T08:19:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 -- 172.21.15.32:0/3179806282 learned_addr learned my addr 172.21.15.32:0/3179806282 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f52152d9640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52100b2600 0x7f52100ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f52162db640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521015a800 0x7f5210159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 -- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52100b2600 msgr2=0x7f52100ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52100b2600 0x7f52100ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 -- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521015a800 msgr2=0x7f5210159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521015a800 0x7f5210159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 -- 172.21.15.32:0/3179806282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208009a70 con 0x7f52100b21c0 2023-10-16T08:19:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5215ada640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 0x7f52101519a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f520c00e870 tx=0x7f520c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:18.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c009510 con 0x7f52100b21c0 2023-10-16T08:19:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c00eec0 con 0x7f52100b21c0 2023-10-16T08:19:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52100caa40 con 0x7f52100b21c0 2023-10-16T08:19:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c00cb40 con 0x7f52100b21c0 2023-10-16T08:19:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52100b3870 con 0x7f52100b21c0 2023-10-16T08:19:18.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52100b3ec0 con 0x7f52100b21c0 2023-10-16T08:19:18.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.416+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f520c00cca0 con 0x7f52100b21c0 2023-10-16T08:19:18.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.420+0000 7f52067fc640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51f40664a0 0x7f51f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.420+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f520c084120 con 0x7f52100b21c0 2023-10-16T08:19:18.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.420+0000 7f52152d9640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51f40664a0 0x7f51f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.420+0000 7f52152d9640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51f40664a0 0x7f51f4068960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5210157f50 tx=0x7f5200009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:18.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.424+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f520c051430 con 0x7f52100b21c0 2023-10-16T08:19:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5210074010 con 0x7f52100b21c0 2023-10-16T08:19:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f52067fc640 1 -- 172.21.15.32:0/3179806282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f520c054a50 con 0x7f52100b21c0 2023-10-16T08:19:18.622 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:18.622 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:18:49.648104+0000","modified":"2023-10-16T08:18:52.672446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/3873719355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3873719355},{"type":"v1","addr":"172.21.15.32:6834","nonce":3873719355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T08:19:18.623 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2023-10-16T08:19:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51f40664a0 msgr2=0x7f51f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51f40664a0 0x7f51f4068960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5210157f50 tx=0x7f5200009040 comp rx=0 tx=0).stop 2023-10-16T08:19:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 msgr2=0x7f52101519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 0x7f52101519a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f520c00e870 tx=0x7f520c00ec70 comp rx=0 tx=0).stop 2023-10-16T08:19:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 shutdown_connections 2023-10-16T08:19:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f521015a800 0x7f5210159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f51f40664a0 0x7f51f4068960 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f52100b2600 0x7f52100ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 --2- 172.21.15.32:0/3179806282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52100b21c0 0x7f52101519a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 >> 172.21.15.32:0/3179806282 conn(0x7f52100cd1c0 msgr2=0x7f52101585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:18.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 shutdown_connections 2023-10-16T08:19:18.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.620+0000 7f5217d3e640 1 -- 172.21.15.32:0/3179806282 wait complete. 2023-10-16T08:19:18.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cf565640 1 Processor -- start 2023-10-16T08:19:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cf565640 1 -- start start 2023-10-16T08:19:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cf565640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80b4fb0 0x7f22c80ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cf565640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c80cdf30 con 0x7f22c80ae960 2023-10-16T08:19:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cf565640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c80cd320 con 0x7f22c80cf0c0 2023-10-16T08:19:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cf565640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8071740 con 0x7f22c80b4fb0 2023-10-16T08:19:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cd301640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f22c80ae960 0x7f22c80ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34844/0 (socket says 172.21.15.32:34844) 2023-10-16T08:19:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cd301640 1 -- 172.21.15.32:0/3428153828 learned_addr learned my addr 172.21.15.32:0/3428153828 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.784+0000 7f22cdb02640 1 --2- 172.21.15.32:0/3428153828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80b4fb0 0x7f22c80ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779639431 0 0) 0x7f22c80cdf30 con 0x7f22c80ae960 2023-10-16T08:19:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b4003530 con 0x7f22c80ae960 2023-10-16T08:19:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491212623 0 0) 0x7f22c80cd320 con 0x7f22c80cf0c0 2023-10-16T08:19:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c80cdf30 con 0x7f22c80cf0c0 2023-10-16T08:19:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3523216948 0 0) 0x7f22b4003530 con 0x7f22c80ae960 2023-10-16T08:19:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22cdb02640 1 -- 172.21.15.32:0/3428153828 >> v1:172.21.15.186:6789/0 conn(0x7f22c80cf0c0 legacy=0x7f22c8159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22cdb02640 1 -- 172.21.15.32:0/3428153828 >> v1:172.21.15.32:6789/0 conn(0x7f22c80ae960 legacy=0x7f22c80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22cdb02640 1 -- 172.21.15.32:0/3428153828 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80cd600 con 0x7f22c80b4fb0 2023-10-16T08:19:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22cdb02640 1 --2- 172.21.15.32:0/3428153828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80b4fb0 0x7f22c80ce3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009dc0 tx=0x7f22c001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e054d3ba40fd0db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4001680 con 0x7f22c80b4fb0 2023-10-16T08:19:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c00a6510 con 0x7f22c80b4fb0 2023-10-16T08:19:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.788+0000 7f22bffff640 1 -- 172.21.15.32:0/3428153828 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00a68a0 con 0x7f22c80b4fb0 2023-10-16T08:19:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- 172.21.15.32:0/3428153828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80b4fb0 msgr2=0x7f22c80ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 --2- 172.21.15.32:0/3428153828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80b4fb0 0x7f22c80ce3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009dc0 tx=0x7f22c001c890 comp rx=0 tx=0).stop 2023-10-16T08:19:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- 172.21.15.32:0/3428153828 shutdown_connections 2023-10-16T08:19:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 --2- 172.21.15.32:0/3428153828 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80b4fb0 0x7f22c80ce3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- 172.21.15.32:0/3428153828 >> 172.21.15.32:0/3428153828 conn(0x7f22c806d310 msgr2=0x7f22c806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- 172.21.15.32:0/3428153828 shutdown_connections 2023-10-16T08:19:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- 172.21.15.32:0/3428153828 wait complete. 2023-10-16T08:19:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 Processor -- start 2023-10-16T08:19:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- start start 2023-10-16T08:19:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 0x7f22c8163cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c80b4fb0 0x7f22c814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80cf0c0 0x7f22c8164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c814f000 con 0x7f22c80b4fb0 2023-10-16T08:19:18.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8162c20 con 0x7f22c80ae960 2023-10-16T08:19:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cf565640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8162f00 con 0x7f22c80cf0c0 2023-10-16T08:19:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cd301640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 0x7f22c8163cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22ccb00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c80b4fb0 0x7f22c814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cd301640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 0x7f22c8163cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.32:51106/0 (socket says 172.21.15.32:51106) 2023-10-16T08:19:18.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cd301640 1 -- 172.21.15.32:0/347037679 learned_addr learned my addr 172.21.15.32:0/347037679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.792+0000 7f22cdb02640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80cf0c0 0x7f22c8164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cd301640 1 -- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80cf0c0 msgr2=0x7f22c8164450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cd301640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80cf0c0 0x7f22c8164450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cd301640 1 -- 172.21.15.32:0/347037679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c80b4fb0 msgr2=0x7f22c814eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cd301640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c80b4fb0 0x7f22c814eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cd301640 1 -- 172.21.15.32:0/347037679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009a70 con 0x7f22c80ae960 2023-10-16T08:19:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22ccb00640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c80b4fb0 0x7f22c814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cd301640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 0x7f22c8163cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c780 tx=0x7f22c400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4005ae0 con 0x7f22c80ae960 2023-10-16T08:19:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4005c60 con 0x7f22c80ae960 2023-10-16T08:19:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4007070 con 0x7f22c80ae960 2023-10-16T08:19:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c81631e0 con 0x7f22c80ae960 2023-10-16T08:19:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c81637a0 con 0x7f22c80ae960 2023-10-16T08:19:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.796+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8163990 con 0x7f22c80ae960 2023-10-16T08:19:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.800+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f22c40071d0 con 0x7f22c80ae960 2023-10-16T08:19:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.800+0000 7f22bdffb640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22ac0664a0 0x7f22ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:18.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.800+0000 7f22ccb00640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22ac0664a0 0x7f22ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.800+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f22c4054710 con 0x7f22c80ae960 2023-10-16T08:19:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.800+0000 7f22ccb00640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22ac0664a0 0x7f22ac068960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22c8152740 tx=0x7f22b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:18.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.804+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f22c4050a50 con 0x7f22c80ae960 2023-10-16T08:19:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.964+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22c8074010 con 0x7f22c80ae960 2023-10-16T08:19:18.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22bdffb640 1 -- 172.21.15.32:0/347037679 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f22c4017070 con 0x7f22c80ae960 2023-10-16T08:19:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:18.970 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:18.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22ac0664a0 msgr2=0x7f22ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22ac0664a0 0x7f22ac068960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22c8152740 tx=0x7f22b8009040 comp rx=0 tx=0).stop 2023-10-16T08:19:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 msgr2=0x7f22c8163cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 0x7f22c8163cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c780 tx=0x7f22c400cc40 comp rx=0 tx=0).stop 2023-10-16T08:19:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 shutdown_connections 2023-10-16T08:19:18.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f22ac0664a0 0x7f22ac068960 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f22c80cf0c0 0x7f22c8164450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c80b4fb0 0x7f22c814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 --2- 172.21.15.32:0/347037679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22c80ae960 0x7f22c8163cc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 >> 172.21.15.32:0/347037679 conn(0x7f22c806d310 msgr2=0x7f22c80ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 shutdown_connections 2023-10-16T08:19:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:18.968+0000 7f22cf565640 1 -- 172.21.15.32:0/347037679 wait complete. 2023-10-16T08:19:18.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:16:10.073833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'testcephfs1_metadata', 'create_time': '2023-10-16T08:18:48.142496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'testcephfs1_data', 'create_time': '2023-10-16T08:18:48.637534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'testcephfs1'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:19:18.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:19:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035eaf7640 1 Processor -- start 2023-10-16T08:19:19.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035eaf7640 1 -- start start 2023-10-16T08:19:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035eaf7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0358069e90 0x7f035806d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035eaf7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035815ae30 con 0x7f035806a270 2023-10-16T08:19:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035eaf7640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03580ca510 con 0x7f03580c26b0 2023-10-16T08:19:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035eaf7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03580cb090 con 0x7f0358069e90 2023-10-16T08:19:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035c893640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0358069e90 0x7f035806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f0357fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f035806a270 0x7f03580c9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34860/0 (socket says 172.21.15.32:34860) 2023-10-16T08:19:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f0357fff640 1 -- 172.21.15.32:0/1347292528 learned_addr learned my addr 172.21.15.32:0/1347292528 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035c893640 1 -- 172.21.15.32:0/1347292528 >> v1:172.21.15.186:6789/0 conn(0x7f03580c26b0 legacy=0x7f03580c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035c893640 1 -- 172.21.15.32:0/1347292528 >> v1:172.21.15.32:6789/0 conn(0x7f035806a270 legacy=0x7f03580c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f035c893640 1 -- 172.21.15.32:0/1347292528 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580ca7f0 con 0x7f0358069e90 2023-10-16T08:19:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.136+0000 7f03577fe640 1 -- 172.21.15.32:0/1347292528 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077347390 0 0) 0x7f035815ae30 con 0x7f035806a270 2023-10-16T08:19:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035c893640 1 --2- 172.21.15.32:0/1347292528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f035806d010 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f034c0097c0 tx=0x7f034c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b400173493e95880 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f03577fe640 1 -- 172.21.15.32:0/1347292528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580c3a20 con 0x7f0358069e90 2023-10-16T08:19:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f03577fe640 1 -- 172.21.15.32:0/1347292528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c0ab650 con 0x7f0358069e90 2023-10-16T08:19:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f03577fe640 1 -- 172.21.15.32:0/1347292528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0ab9e0 con 0x7f0358069e90 2023-10-16T08:19:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 -- 172.21.15.32:0/1347292528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 msgr2=0x7f035806d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 --2- 172.21.15.32:0/1347292528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f035806d010 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f034c0097c0 tx=0x7f034c01c450 comp rx=0 tx=0).stop 2023-10-16T08:19:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 -- 172.21.15.32:0/1347292528 shutdown_connections 2023-10-16T08:19:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 --2- 172.21.15.32:0/1347292528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f035806d010 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 -- 172.21.15.32:0/1347292528 >> 172.21.15.32:0/1347292528 conn(0x7f03580b0da0 msgr2=0x7f03580ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 -- 172.21.15.32:0/1347292528 shutdown_connections 2023-10-16T08:19:19.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.140+0000 7f035eaf7640 1 -- 172.21.15.32:0/1347292528 wait complete. 2023-10-16T08:19:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 Processor -- start 2023-10-16T08:19:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 -- start start 2023-10-16T08:19:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f0358151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f035806a270 0x7f03580cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03580c26b0 0x7f03580c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580cd910 con 0x7f0358069e90 2023-10-16T08:19:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580ccd00 con 0x7f03580c26b0 2023-10-16T08:19:19.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580ccfe0 con 0x7f035806a270 2023-10-16T08:19:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f0358151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f0358151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48764/0 (socket says 172.21.15.32:48764) 2023-10-16T08:19:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 -- 172.21.15.32:0/331458965 learned_addr learned my addr 172.21.15.32:0/331458965 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f0357fff640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f035806a270 0x7f03580cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035d094640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03580c26b0 0x7f03580c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 -- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f035806a270 msgr2=0x7f03580cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f035806a270 0x7f03580cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 -- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03580c26b0 msgr2=0x7f03580c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03580c26b0 0x7f03580c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 -- 172.21.15.32:0/331458965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c009420 con 0x7f0358069e90 2023-10-16T08:19:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035d094640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03580c26b0 0x7f03580c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f0357fff640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f035806a270 0x7f03580cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035c893640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f0358151d40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f034c0ad040 tx=0x7f034c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0b5810 con 0x7f0358069e90 2023-10-16T08:19:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c0b5970 con 0x7f0358069e90 2023-10-16T08:19:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0b5d20 con 0x7f0358069e90 2023-10-16T08:19:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03580cd2c0 con 0x7f0358069e90 2023-10-16T08:19:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.144+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03580cc960 con 0x7f0358069e90 2023-10-16T08:19:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.148+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035804e170 con 0x7f0358069e90 2023-10-16T08:19:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.148+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f034c0bf410 con 0x7f0358069e90 2023-10-16T08:19:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.148+0000 7f03557fa640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f033c0664d0 0x7f033c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.148+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f034c11ed30 con 0x7f0358069e90 2023-10-16T08:19:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.148+0000 7f0357fff640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f033c0664d0 0x7f033c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.148+0000 7f0357fff640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f033c0664d0 0x7f033c068990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03580ad380 tx=0x7f0344007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.152+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f034c0ebfc0 con 0x7f0358069e90 2023-10-16T08:19:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.308+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03580c2090 con 0x7f0358069e90 2023-10-16T08:19:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f03557fa640 1 -- 172.21.15.32:0/331458965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f034c0ef5e0 con 0x7f0358069e90 2023-10-16T08:19:19.313 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-10-16T08:19:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f033c0664d0 msgr2=0x7f033c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f033c0664d0 0x7f033c068990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03580ad380 tx=0x7f0344007c60 comp rx=0 tx=0).stop 2023-10-16T08:19:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 msgr2=0x7f0358151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f0358151d40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f034c0ad040 tx=0x7f034c01ac90 comp rx=0 tx=0).stop 2023-10-16T08:19:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 shutdown_connections 2023-10-16T08:19:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03580c26b0 0x7f03580c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f033c0664d0 0x7f033c068990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f035806a270 0x7f03580cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 --2- 172.21.15.32:0/331458965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0358069e90 0x7f0358151d40 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 >> 172.21.15.32:0/331458965 conn(0x7f03580b0da0 msgr2=0x7f035815a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 shutdown_connections 2023-10-16T08:19:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.312+0000 7f035eaf7640 1 -- 172.21.15.32:0/331458965 wait complete. 2023-10-16T08:19:19.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_metadata pg_num 2023-10-16T08:19:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.476+0000 7fdfb0605640 1 Processor -- start 2023-10-16T08:19:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.476+0000 7fdfb0605640 1 -- start start 2023-10-16T08:19:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfb0605640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 0x7fdfa806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfb0605640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa80b4340 con 0x7fdfa80b3f00 2023-10-16T08:19:19.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfb0605640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa80b4600 con 0x7fdfa8159900 2023-10-16T08:19:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfb0605640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80cd740 con 0x7fdfa806a5a0 2023-10-16T08:19:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfaeba2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdfa80b3f00 0x7fdfa80c9b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34874/0 (socket says 172.21.15.32:34874) 2023-10-16T08:19:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfaeba2640 1 -- 172.21.15.32:0/3704561972 learned_addr learned my addr 172.21.15.32:0/3704561972 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfadba0640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 0x7fdfa806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146968650 0 0) 0x7fdfa80b4340 con 0x7fdfa80b3f00 2023-10-16T08:19:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf94003530 con 0x7fdfa80b3f00 2023-10-16T08:19:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732344152 0 0) 0x7fdfa80b4600 con 0x7fdfa8159900 2023-10-16T08:19:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfa80b4340 con 0x7fdfa8159900 2023-10-16T08:19:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708628095 0 0) 0x7fdf94003530 con 0x7fdfa80b3f00 2023-10-16T08:19:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 msgr2=0x7fdfa806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 0x7fdfa806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 >> v1:172.21.15.186:6789/0 conn(0x7fdfa8159900 legacy=0x7fdfa8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa80c8dd0 con 0x7fdfa80b3f00 2023-10-16T08:19:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fdfa0001980 con 0x7fdfa80b3f00 2023-10-16T08:19:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 >> v1:172.21.15.32:6789/0 conn(0x7fdfa80b3f00 legacy=0x7fdfa80c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdf94003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf940047d0 0x7fdf94004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 0x7fdf94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80cd740 con 0x7fdf94003b40 2023-10-16T08:19:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9400dc50 con 0x7fdf940047d0 2023-10-16T08:19:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9400de10 con 0x7fdfa806a5a0 2023-10-16T08:19:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdf94003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfaeba2640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 0x7fdf94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfadba0640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf940047d0 0x7fdf94004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfae3a1640 1 -- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 msgr2=0x7fdf94005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa806a5a0 0x7fdf94005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfae3a1640 1 -- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf940047d0 msgr2=0x7fdf94004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf940047d0 0x7fdf94004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.480+0000 7fdfae3a1640 1 -- 172.21.15.32:0/3704561972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94003530 con 0x7fdf94003b40 2023-10-16T08:19:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdf94003f40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40018d0 tx=0x7fdfa401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6023135ebd0e9fac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40052d0 con 0x7fdf94003b40 2023-10-16T08:19:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4005450 con 0x7fdf94003b40 2023-10-16T08:19:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfad39f640 1 -- 172.21.15.32:0/3704561972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4003840 con 0x7fdf94003b40 2023-10-16T08:19:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfb0605640 1 -- 172.21.15.32:0/3704561972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 msgr2=0x7fdf94003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfb0605640 1 --2- 172.21.15.32:0/3704561972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdf94003f40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40018d0 tx=0x7fdfa401c530 comp rx=0 tx=0).stop 2023-10-16T08:19:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfaeba2640 1 -- 172.21.15.32:0/3704561972 reap_dead start 2023-10-16T08:19:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfb0605640 1 -- 172.21.15.32:0/3704561972 shutdown_connections 2023-10-16T08:19:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfb0605640 1 -- 172.21.15.32:0/3704561972 >> 172.21.15.32:0/3704561972 conn(0x7fdfa80b0da0 msgr2=0x7fdfa80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfb0605640 1 -- 172.21.15.32:0/3704561972 shutdown_connections 2023-10-16T08:19:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.484+0000 7fdfb0605640 1 -- 172.21.15.32:0/3704561972 wait complete. 2023-10-16T08:19:19.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 Processor -- start 2023-10-16T08:19:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 -- start start 2023-10-16T08:19:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdfa80c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa80c5850 0x7fdfa8154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa80701f0 0x7fdfa8070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8070bb0 con 0x7fdf94003b40 2023-10-16T08:19:19.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8155220 con 0x7fdfa80c5850 2023-10-16T08:19:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80c5cd0 con 0x7fdfa80701f0 2023-10-16T08:19:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdfa80c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdfa80c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48776/0 (socket says 172.21.15.32:48776) 2023-10-16T08:19:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 -- 172.21.15.32:0/2828118744 learned_addr learned my addr 172.21.15.32:0/2828118744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfadba0640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa80c5850 0x7fdfa8154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfaeba2640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa80701f0 0x7fdfa8070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 -- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa80701f0 msgr2=0x7fdfa8070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa80701f0 0x7fdfa8070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 -- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa80c5850 msgr2=0x7fdfa8154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa80c5850 0x7fdfa8154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 -- 172.21.15.32:0/2828118744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa80c5fa0 con 0x7fdf94003b40 2023-10-16T08:19:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfaeba2640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa80701f0 0x7fdfa8070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfadba0640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa80c5850 0x7fdfa8154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfae3a1640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdfa80c7d20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40018d0 tx=0x7fdf94005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40050d0 con 0x7fdf94003b40 2023-10-16T08:19:19.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4005230 con 0x7fdf94003b40 2023-10-16T08:19:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4006a00 con 0x7fdf94003b40 2023-10-16T08:19:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa80c6280 con 0x7fdf94003b40 2023-10-16T08:19:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.488+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa80c6840 con 0x7fdf94003b40 2023-10-16T08:19:19.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.492+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa804e170 con 0x7fdf94003b40 2023-10-16T08:19:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.492+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdfa4006b60 con 0x7fdf94003b40 2023-10-16T08:19:19.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.492+0000 7fdf9effd640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf8c0664a0 0x7fdf8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.492+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fdfa411d840 con 0x7fdf94003b40 2023-10-16T08:19:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.492+0000 7fdfadba0640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf8c0664a0 0x7fdf8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.492+0000 7fdfadba0640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf8c0664a0 0x7fdf8c068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdfa81f4aa0 tx=0x7fdf98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.496+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdfa40eaac0 con 0x7fdf94003b40 2023-10-16T08:19:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfa80afd80 con 0x7fdf94003b40 2023-10-16T08:19:19.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdf9effd640 1 -- 172.21.15.32:0/2828118744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_metadata", "var": "pg_num"}]=0 v54) v1 ==== 109+0+11 (secure 0 0 0) 0x7fdfa40ee0e0 con 0x7fdf94003b40 2023-10-16T08:19:19.655 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-10-16T08:19:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf8c0664a0 msgr2=0x7fdf8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf8c0664a0 0x7fdf8c068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdfa81f4aa0 tx=0x7fdf98008040 comp rx=0 tx=0).stop 2023-10-16T08:19:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 msgr2=0x7fdfa80c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdfa80c7d20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40018d0 tx=0x7fdf94005800 comp rx=0 tx=0).stop 2023-10-16T08:19:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 shutdown_connections 2023-10-16T08:19:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdf8c0664a0 0x7fdf8c068960 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdfa80701f0 0x7fdfa8070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfa80c5850 0x7fdfa8154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 --2- 172.21.15.32:0/2828118744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf94003b40 0x7fdfa80c7d20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 >> 172.21.15.32:0/2828118744 conn(0x7fdfa80b0da0 msgr2=0x7fdfa81581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 shutdown_connections 2023-10-16T08:19:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.652+0000 7fdfb0605640 1 -- 172.21.15.32:0/2828118744 wait complete. 2023-10-16T08:19:19.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get testcephfs1_data pg_num 2023-10-16T08:19:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.820+0000 7f39eb0d6640 1 Processor -- start 2023-10-16T08:19:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39eb0d6640 1 -- start start 2023-10-16T08:19:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39eb0d6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f39e415cba0 0x7f39e40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39eb0d6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39e40d0210 0x7f39e40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39eb0d6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e40d1670 con 0x7f39e40b84e0 2023-10-16T08:19:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39eb0d6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e40d1460 con 0x7f39e40d0210 2023-10-16T08:19:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39eb0d6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e40d1950 con 0x7f39e415cba0 2023-10-16T08:19:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e9673640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39e40d0210 0x7f39e40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39e40b84e0 0x7f39e40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34886/0 (socket says 172.21.15.32:34886) 2023-10-16T08:19:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e3fff640 1 -- 172.21.15.32:0/1429929000 learned_addr learned my addr 172.21.15.32:0/1429929000 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e8e72640 1 --2- 172.21.15.32:0/1429929000 >> v2:172.21.15.186:3300/0 conn(0x7f39e415cba0 0x7f39e40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e9673640 1 -- 172.21.15.32:0/1429929000 >> v2:172.21.15.186:3300/0 conn(0x7f39e415cba0 msgr2=0x7f39e40d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e9673640 1 --2- 172.21.15.32:0/1429929000 >> v2:172.21.15.186:3300/0 conn(0x7f39e415cba0 0x7f39e40d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e9673640 1 -- 172.21.15.32:0/1429929000 >> v1:172.21.15.32:6789/0 conn(0x7f39e40b84e0 legacy=0x7f39e40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e37fe640 1 -- 172.21.15.32:0/1429929000 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355017688 0 0) 0x7f39e40d1670 con 0x7f39e40b84e0 2023-10-16T08:19:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.824+0000 7f39e9673640 1 -- 172.21.15.32:0/1429929000 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40d1c30 con 0x7f39e40d0210 2023-10-16T08:19:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39e9673640 1 --2- 172.21.15.32:0/1429929000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40d0210 0x7f39e40b5cf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f39e40b4890 tx=0x7f39dc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bcfc50ebc0705d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39e37fe640 1 -- 172.21.15.32:0/1429929000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0a5940 con 0x7f39e40d0210 2023-10-16T08:19:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39e37fe640 1 -- 172.21.15.32:0/1429929000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc0a5aa0 con 0x7f39e40d0210 2023-10-16T08:19:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39e37fe640 1 -- 172.21.15.32:0/1429929000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0a5e30 con 0x7f39e40d0210 2023-10-16T08:19:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 -- 172.21.15.32:0/1429929000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40d0210 msgr2=0x7f39e40b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/1429929000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40d0210 0x7f39e40b5cf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f39e40b4890 tx=0x7f39dc01c880 comp rx=0 tx=0).stop 2023-10-16T08:19:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 -- 172.21.15.32:0/1429929000 shutdown_connections 2023-10-16T08:19:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/1429929000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40d0210 0x7f39e40b5cf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/1429929000 >> v2:172.21.15.186:3300/0 conn(0x7f39e415cba0 0x7f39e40d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 -- 172.21.15.32:0/1429929000 >> 172.21.15.32:0/1429929000 conn(0x7f39e40719c0 msgr2=0x7f39e4071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 -- 172.21.15.32:0/1429929000 shutdown_connections 2023-10-16T08:19:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 -- 172.21.15.32:0/1429929000 wait complete. 2023-10-16T08:19:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 Processor -- start 2023-10-16T08:19:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.828+0000 7f39eb0d6640 1 -- start start 2023-10-16T08:19:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 0x7f39e40c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e40d0210 0x7f39e40c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e415cba0 0x7f39e415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e40c48e0 con 0x7f39e40b84e0 2023-10-16T08:19:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e40c8eb0 con 0x7f39e40d0210 2023-10-16T08:19:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e40c3d30 con 0x7f39e415cba0 2023-10-16T08:19:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 0x7f39e40c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 0x7f39e40c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48790/0 (socket says 172.21.15.32:48790) 2023-10-16T08:19:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 -- 172.21.15.32:0/2587650043 learned_addr learned my addr 172.21.15.32:0/2587650043 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 -- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e415cba0 msgr2=0x7f39e415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e415cba0 0x7f39e415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 -- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e40d0210 msgr2=0x7f39e40c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e40d0210 0x7f39e40c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 -- 172.21.15.32:0/2587650043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc009a70 con 0x7f39e40b84e0 2023-10-16T08:19:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e8e72640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 0x7f39e40c4d00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f39d800e870 tx=0x7f39d800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8009510 con 0x7f39e40b84e0 2023-10-16T08:19:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d800eec0 con 0x7f39e40b84e0 2023-10-16T08:19:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800cb40 con 0x7f39e40b84e0 2023-10-16T08:19:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e40c3f40 con 0x7f39e40b84e0 2023-10-16T08:19:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e40c8a80 con 0x7f39e40b84e0 2023-10-16T08:19:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.832+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e404e170 con 0x7f39e40b84e0 2023-10-16T08:19:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.836+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39d800cca0 con 0x7f39e40b84e0 2023-10-16T08:19:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.836+0000 7f39e17fa640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c80664a0 0x7f39c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.836+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f39d80841b0 con 0x7f39e40b84e0 2023-10-16T08:19:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.836+0000 7f39e3fff640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c80664a0 0x7f39c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.836+0000 7f39e3fff640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c80664a0 0x7f39c8068960 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39d00011f0 tx=0x7f39d0007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:19.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:19.840+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39d8051430 con 0x7f39e40b84e0 2023-10-16T08:19:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.000+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"} v 0) v1 -- 0x7f39e4153900 con 0x7f39e40b84e0 2023-10-16T08:19:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39e17fa640 1 -- 172.21.15.32:0/2587650043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testcephfs1_data", "var": "pg_num"}]=0 v54) v1 ==== 105+0+11 (secure 0 0 0) 0x7f39d8054a50 con 0x7f39e40b84e0 2023-10-16T08:19:20.005 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-10-16T08:19:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c80664a0 msgr2=0x7f39c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c80664a0 0x7f39c8068960 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39d00011f0 tx=0x7f39d0007930 comp rx=0 tx=0).stop 2023-10-16T08:19:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 msgr2=0x7f39e40c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 0x7f39e40c4d00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f39d800e870 tx=0x7f39d800ec70 comp rx=0 tx=0).stop 2023-10-16T08:19:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 shutdown_connections 2023-10-16T08:19:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39c80664a0 0x7f39c8068960 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39e415cba0 0x7f39e415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39e40d0210 0x7f39e40c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 --2- 172.21.15.32:0/2587650043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39e40b84e0 0x7f39e40c4d00 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 >> 172.21.15.32:0/2587650043 conn(0x7f39e40719c0 msgr2=0x7f39e40b6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 shutdown_connections 2023-10-16T08:19:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.004+0000 7f39eb0d6640 1 -- 172.21.15.32:0/2587650043 wait complete. 2023-10-16T08:19:20.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:19:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 Processor -- start 2023-10-16T08:19:20.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 -- start start 2023-10-16T08:19:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb6015cb90 0x7fdb600cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb600d15d0 0x7fdb600c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb600ce770 con 0x7fdb600d3010 2023-10-16T08:19:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600d0600 con 0x7fdb6015cb90 2023-10-16T08:19:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb67dc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c6450 con 0x7fdb600d15d0 2023-10-16T08:19:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb6015cb90 0x7fdb600cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb6015cb90 0x7fdb600cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48804/0 (socket says 172.21.15.32:48804) 2023-10-16T08:19:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 -- 172.21.15.32:0/1813368251 learned_addr learned my addr 172.21.15.32:0/1813368251 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 -- 172.21.15.32:0/1813368251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb600d15d0 msgr2=0x7fdb600c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb66362640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb600d15d0 0x7fdb600c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb600d15d0 0x7fdb600c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 -- 172.21.15.32:0/1813368251 >> v1:172.21.15.186:6789/0 conn(0x7fdb600d3010 legacy=0x7fdb600d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb65b61640 1 -- 172.21.15.32:0/1813368251 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600d08a0 con 0x7fdb6015cb90 2023-10-16T08:19:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.168+0000 7fdb66362640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb600d15d0 0x7fdb600c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb65b61640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6015cb90 0x7fdb600cfb10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdb6008ba70 tx=0x7fdb5c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b40b55d4166c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb64b5f640 1 -- 172.21.15.32:0/1813368251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0ac620 con 0x7fdb6015cb90 2023-10-16T08:19:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb64b5f640 1 -- 172.21.15.32:0/1813368251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c0ac780 con 0x7fdb6015cb90 2023-10-16T08:19:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb64b5f640 1 -- 172.21.15.32:0/1813368251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0acb10 con 0x7fdb6015cb90 2023-10-16T08:19:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 -- 172.21.15.32:0/1813368251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6015cb90 msgr2=0x7fdb600cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6015cb90 0x7fdb600cfb10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdb6008ba70 tx=0x7fdb5c01c400 comp rx=0 tx=0).stop 2023-10-16T08:19:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 -- 172.21.15.32:0/1813368251 shutdown_connections 2023-10-16T08:19:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb600d15d0 0x7fdb600c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/1813368251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb6015cb90 0x7fdb600cfb10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 -- 172.21.15.32:0/1813368251 >> 172.21.15.32:0/1813368251 conn(0x7fdb6009bc50 msgr2=0x7fdb600d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 -- 172.21.15.32:0/1813368251 shutdown_connections 2023-10-16T08:19:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.172+0000 7fdb67dc5640 1 -- 172.21.15.32:0/1813368251 wait complete. 2023-10-16T08:19:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 Processor -- start 2023-10-16T08:19:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- start start 2023-10-16T08:19:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb600d15d0 0x7fdb600b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 0x7fdb600cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb6015cb90 0x7fdb6014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600b3cd0 con 0x7fdb600d3010 2023-10-16T08:19:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6014f160 con 0x7fdb600d15d0 2023-10-16T08:19:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600b56c0 con 0x7fdb6015cb90 2023-10-16T08:19:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 0x7fdb600cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 0x7fdb600cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48808/0 (socket says 172.21.15.32:48808) 2023-10-16T08:19:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 -- 172.21.15.32:0/924862361 learned_addr learned my addr 172.21.15.32:0/924862361 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65b61640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb600d15d0 0x7fdb600b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb66362640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb6015cb90 0x7fdb6014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 -- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb6015cb90 msgr2=0x7fdb6014ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb6015cb90 0x7fdb6014ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 -- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb600d15d0 msgr2=0x7fdb600b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb600d15d0 0x7fdb600b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 -- 172.21.15.32:0/924862361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c009480 con 0x7fdb600d3010 2023-10-16T08:19:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb66362640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb6015cb90 0x7fdb6014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb65360640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 0x7fdb600cabf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c450 tx=0x7fdb5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50012830 con 0x7fdb600d3010 2023-10-16T08:19:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb500129b0 con 0x7fdb600d3010 2023-10-16T08:19:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50012d60 con 0x7fdb600d3010 2023-10-16T08:19:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb600b59a0 con 0x7fdb600d3010 2023-10-16T08:19:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb600b5f60 con 0x7fdb600d3010 2023-10-16T08:19:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.176+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb600b75b0 con 0x7fdb600d3010 2023-10-16T08:19:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.180+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdb5001c460 con 0x7fdb600d3010 2023-10-16T08:19:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.180+0000 7fdb567fc640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb440664a0 0x7fdb44068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.180+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fdb50083f30 con 0x7fdb600d3010 2023-10-16T08:19:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.180+0000 7fdb65b61640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb440664a0 0x7fdb44068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.180+0000 7fdb65b61640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb440664a0 0x7fdb44068960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0ae000 tx=0x7fdb5c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.184+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdb50051120 con 0x7fdb600d3010 2023-10-16T08:19:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.376+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb60074010 con 0x7fdb600d3010 2023-10-16T08:19:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.376+0000 7fdb567fc640 1 -- 172.21.15.32:0/924862361 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fdb50054740 con 0x7fdb600d3010 2023-10-16T08:19:20.380 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:20.380 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:18:49.648104+0000","modified":"2023-10-16T08:18:52.672446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/3873719355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3873719355},{"type":"v1","addr":"172.21.15.32:6834","nonce":3873719355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T08:19:20.380 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2023-10-16T08:19:20.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb440664a0 msgr2=0x7fdb44068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb440664a0 0x7fdb44068960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0ae000 tx=0x7fdb5c0a7000 comp rx=0 tx=0).stop 2023-10-16T08:19:20.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 msgr2=0x7fdb600cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 0x7fdb600cabf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c450 tx=0x7fdb5000c910 comp rx=0 tx=0).stop 2023-10-16T08:19:20.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 shutdown_connections 2023-10-16T08:19:20.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fdb440664a0 0x7fdb44068960 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fdb6015cb90 0x7fdb6014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb600d3010 0x7fdb600cabf0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 --2- 172.21.15.32:0/924862361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb600d15d0 0x7fdb600b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 >> 172.21.15.32:0/924862361 conn(0x7fdb6009bc50 msgr2=0x7fdb6015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:20.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 shutdown_connections 2023-10-16T08:19:20.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.380+0000 7fdb67dc5640 1 -- 172.21.15.32:0/924862361 wait complete. 2023-10-16T08:19:20.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.540+0000 7f3f8e53e640 1 Processor -- start 2023-10-16T08:19:20.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.540+0000 7f3f8e53e640 1 -- start start 2023-10-16T08:19:20.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8e53e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f880b2ad0 0x7f3f8806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8e53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f8806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8e53e640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f3f880ce310 0x7f3f880ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8e53e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cece0 con 0x7f3f880b2ad0 2023-10-16T08:19:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8e53e640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f88160580 con 0x7f3f880ce310 2023-10-16T08:19:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8e53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815d4e0 con 0x7f3f8815cf90 2023-10-16T08:19:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f880b2ad0 0x7f3f8806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f880b2ad0 0x7f3f8806b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48824/0 (socket says 172.21.15.32:48824) 2023-10-16T08:19:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 -- 172.21.15.32:0/1409360686 learned_addr learned my addr 172.21.15.32:0/1409360686 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f877fe640 1 --2- 172.21.15.32:0/1409360686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f8806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8cadb640 1 --2- 172.21.15.32:0/1409360686 >> v2:172.21.15.186:3300/0 conn(0x7f3f880ce310 0x7f3f880ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 -- 172.21.15.32:0/1409360686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 msgr2=0x7f3f8806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 --2- 172.21.15.32:0/1409360686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f8806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 -- 172.21.15.32:0/1409360686 >> v2:172.21.15.186:3300/0 conn(0x7f3f880ce310 msgr2=0x7f3f880ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:20.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 --2- 172.21.15.32:0/1409360686 >> v2:172.21.15.186:3300/0 conn(0x7f3f880ce310 0x7f3f880ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f87fff640 1 -- 172.21.15.32:0/1409360686 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8815d7c0 con 0x7f3f880b2ad0 2023-10-16T08:19:20.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.544+0000 7f3f8cadb640 1 --2- 172.21.15.32:0/1409360686 >> v2:172.21.15.186:3300/0 conn(0x7f3f880ce310 0x7f3f880ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f87fff640 1 --2- 172.21.15.32:0/1409360686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880b2ad0 0x7f3f8806b4b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806c620 tx=0x7f3f7801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa070124bbc4700 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f86ffd640 1 -- 172.21.15.32:0/1409360686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780aa5f0 con 0x7f3f880b2ad0 2023-10-16T08:19:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f86ffd640 1 -- 172.21.15.32:0/1409360686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f780aa750 con 0x7f3f880b2ad0 2023-10-16T08:19:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f86ffd640 1 -- 172.21.15.32:0/1409360686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780aaae0 con 0x7f3f880b2ad0 2023-10-16T08:19:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 -- 172.21.15.32:0/1409360686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880b2ad0 msgr2=0x7f3f8806b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/1409360686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880b2ad0 0x7f3f8806b4b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806c620 tx=0x7f3f7801c4d0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 -- 172.21.15.32:0/1409360686 shutdown_connections 2023-10-16T08:19:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/1409360686 >> v2:172.21.15.186:3300/0 conn(0x7f3f880ce310 0x7f3f880ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/1409360686 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f8806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/1409360686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880b2ad0 0x7f3f8806b4b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 -- 172.21.15.32:0/1409360686 >> 172.21.15.32:0/1409360686 conn(0x7f3f880cd1c0 msgr2=0x7f3f880cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 -- 172.21.15.32:0/1409360686 shutdown_connections 2023-10-16T08:19:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 -- 172.21.15.32:0/1409360686 wait complete. 2023-10-16T08:19:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 Processor -- start 2023-10-16T08:19:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.548+0000 7f3f8e53e640 1 -- start start 2023-10-16T08:19:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f880b2ad0 0x7f3f881539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 0x7f3f88158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f88158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880b5170 con 0x7f3f880ce310 2023-10-16T08:19:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880b5fb0 con 0x7f3f880b2ad0 2023-10-16T08:19:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88153f00 con 0x7f3f8815cf90 2023-10-16T08:19:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 0x7f3f88158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 0x7f3f88158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48838/0 (socket says 172.21.15.32:48838) 2023-10-16T08:19:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 -- 172.21.15.32:0/4080139960 learned_addr learned my addr 172.21.15.32:0/4080139960 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 -- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 msgr2=0x7f3f88158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f88158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 -- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f880b2ad0 msgr2=0x7f3f881539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f880b2ad0 0x7f3f881539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 -- 172.21.15.32:0/4080139960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780094d0 con 0x7f3f880ce310 2023-10-16T08:19:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f877fe640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 0x7f3f88158a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d620 tx=0x7f3f7c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00b620 con 0x7f3f880ce310 2023-10-16T08:19:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00b780 con 0x7f3f880ce310 2023-10-16T08:19:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00bb10 con 0x7f3f880ce310 2023-10-16T08:19:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f880b53a0 con 0x7f3f880ce310 2023-10-16T08:19:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f880b5960 con 0x7f3f880ce310 2023-10-16T08:19:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f88074010 con 0x7f3f880ce310 2023-10-16T08:19:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.552+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f7c00bc70 con 0x7f3f880ce310 2023-10-16T08:19:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.556+0000 7f3f84ff9640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3f740664d0 0x7f3f74068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.556+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f3f7c084460 con 0x7f3f880ce310 2023-10-16T08:19:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.556+0000 7f3f87fff640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3f740664d0 0x7f3f74068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.556+0000 7f3f87fff640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3f740664d0 0x7f3f74068990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3f780ae000 tx=0x7f3f78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.560+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f7c0516e0 con 0x7f3f880ce310 2023-10-16T08:19:20.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.716+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f880cddb0 con 0x7f3f880ce310 2023-10-16T08:19:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.716+0000 7f3f84ff9640 1 -- 172.21.15.32:0/4080139960 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f3f7c054d00 con 0x7f3f880ce310 2023-10-16T08:19:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3f740664d0 msgr2=0x7f3f74068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3f740664d0 0x7f3f74068990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3f780ae000 tx=0x7f3f78008040 comp rx=0 tx=0).stop 2023-10-16T08:19:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 msgr2=0x7f3f88158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 0x7f3f88158a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d620 tx=0x7f3f7c00dae0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 shutdown_connections 2023-10-16T08:19:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f3f740664d0 0x7f3f74068990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3f8815cf90 0x7f3f88158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f880ce310 0x7f3f88158a10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 --2- 172.21.15.32:0/4080139960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f880b2ad0 0x7f3f881539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 >> 172.21.15.32:0/4080139960 conn(0x7f3f880cd1c0 msgr2=0x7f3f8815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 shutdown_connections 2023-10-16T08:19:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.720+0000 7f3f8e53e640 1 -- 172.21.15.32:0/4080139960 wait complete. 2023-10-16T08:19:20.742 INFO:tasks.cephfs.filesystem:Destroying file system testcephfs1 and related pools 2023-10-16T08:19:20.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:19:20.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.884+0000 7fac8e446640 1 Processor -- start 2023-10-16T08:19:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.884+0000 7fac8e446640 1 -- start start 2023-10-16T08:19:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.884+0000 7fac8e446640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac880d15d0 0x7fac880cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.884+0000 7fac8e446640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fac880d3010 0x7fac880c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.884+0000 7fac8e446640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac880ce770 con 0x7fac8815cb90 2023-10-16T08:19:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac8e446640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac880d0600 con 0x7fac880d15d0 2023-10-16T08:19:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac8e446640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac880c6450 con 0x7fac880d3010 2023-10-16T08:19:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac880d15d0 0x7fac880cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac880d15d0 0x7fac880cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48850/0 (socket says 172.21.15.32:48850) 2023-10-16T08:19:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 -- 172.21.15.32:0/4202400904 learned_addr learned my addr 172.21.15.32:0/4202400904 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac8c9e3640 1 --2- 172.21.15.32:0/4202400904 >> v2:172.21.15.186:3300/0 conn(0x7fac880d3010 0x7fac880c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 -- 172.21.15.32:0/4202400904 >> v2:172.21.15.186:3300/0 conn(0x7fac880d3010 msgr2=0x7fac880c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 --2- 172.21.15.32:0/4202400904 >> v2:172.21.15.186:3300/0 conn(0x7fac880d3010 0x7fac880c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 -- 172.21.15.32:0/4202400904 >> v1:172.21.15.186:6789/0 conn(0x7fac8815cb90 legacy=0x7fac880d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 -- 172.21.15.32:0/4202400904 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac880d08a0 con 0x7fac880d15d0 2023-10-16T08:19:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac877fe640 1 --2- 172.21.15.32:0/4202400904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac880cfb10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fac88072590 tx=0x7fac7c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a93d386b41fbc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac86ffd640 1 -- 172.21.15.32:0/4202400904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0ab640 con 0x7fac880d15d0 2023-10-16T08:19:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac86ffd640 1 -- 172.21.15.32:0/4202400904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c0a7070 con 0x7fac880d15d0 2023-10-16T08:19:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.888+0000 7fac86ffd640 1 -- 172.21.15.32:0/4202400904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0ab9c0 con 0x7fac880d15d0 2023-10-16T08:19:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- 172.21.15.32:0/4202400904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 msgr2=0x7fac880cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 --2- 172.21.15.32:0/4202400904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac880cfb10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fac88072590 tx=0x7fac7c01c440 comp rx=0 tx=0).stop 2023-10-16T08:19:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- 172.21.15.32:0/4202400904 shutdown_connections 2023-10-16T08:19:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 --2- 172.21.15.32:0/4202400904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac880cfb10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 --2- 172.21.15.32:0/4202400904 >> v2:172.21.15.186:3300/0 conn(0x7fac880d3010 0x7fac880c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- 172.21.15.32:0/4202400904 >> 172.21.15.32:0/4202400904 conn(0x7fac8809bc50 msgr2=0x7fac880d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- 172.21.15.32:0/4202400904 shutdown_connections 2023-10-16T08:19:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- 172.21.15.32:0/4202400904 wait complete. 2023-10-16T08:19:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 Processor -- start 2023-10-16T08:19:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- start start 2023-10-16T08:19:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac8806bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac880d3010 0x7fac880cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac8815cb90 0x7fac880cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88157ba0 con 0x7fac880d15d0 2023-10-16T08:19:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88157e80 con 0x7fac8815cb90 2023-10-16T08:19:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8e446640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac88158160 con 0x7fac880d3010 2023-10-16T08:19:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac8806bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac8806bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48856/0 (socket says 172.21.15.32:48856) 2023-10-16T08:19:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac87fff640 1 -- 172.21.15.32:0/2082916858 learned_addr learned my addr 172.21.15.32:0/2082916858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac8c9e3640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac8815cb90 0x7fac880cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.892+0000 7fac877fe640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac880d3010 0x7fac880cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac87fff640 1 -- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac880d3010 msgr2=0x7fac880cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac87fff640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac880d3010 0x7fac880cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac87fff640 1 -- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac8815cb90 msgr2=0x7fac880cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac87fff640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac8815cb90 0x7fac880cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac87fff640 1 -- 172.21.15.32:0/2082916858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c0094d0 con 0x7fac880d15d0 2023-10-16T08:19:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac87fff640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac8806bcc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fac7800c400 tx=0x7fac7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac780052d0 con 0x7fac880d15d0 2023-10-16T08:19:20.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac78005430 con 0x7fac880d15d0 2023-10-16T08:19:20.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78003890 con 0x7fac880d15d0 2023-10-16T08:19:20.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8809e2c0 con 0x7fac880d15d0 2023-10-16T08:19:20.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8809c890 con 0x7fac880d15d0 2023-10-16T08:19:20.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac88074010 con 0x7fac880d15d0 2023-10-16T08:19:20.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac780039f0 con 0x7fac880d15d0 2023-10-16T08:19:20.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac84ff9640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fac740664d0 0x7fac74068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fac780835b0 con 0x7fac880d15d0 2023-10-16T08:19:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.896+0000 7fac877fe640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fac740664d0 0x7fac74068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.900+0000 7fac877fe640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fac740664d0 0x7fac74068990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0af000 tx=0x7fac7c0b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:20.900+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac780508e0 con 0x7fac880d15d0 2023-10-16T08:19:21.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.092+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac8809e770 con 0x7fac880d15d0 2023-10-16T08:19:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.092+0000 7fac84ff9640 1 -- 172.21.15.32:0/2082916858 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fac78053f00 con 0x7fac880d15d0 2023-10-16T08:19:21.097 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:21.097 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:18:49.648104+0000","modified":"2023-10-16T08:18:52.672446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/3873719355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3873719355},{"type":"v1","addr":"172.21.15.32:6834","nonce":3873719355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T08:19:21.098 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2023-10-16T08:19:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fac740664d0 msgr2=0x7fac74068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fac740664d0 0x7fac74068990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0af000 tx=0x7fac7c0b6040 comp rx=0 tx=0).stop 2023-10-16T08:19:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 msgr2=0x7fac8806bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac8806bcc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fac7800c400 tx=0x7fac7800c8c0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 shutdown_connections 2023-10-16T08:19:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac8815cb90 0x7fac880cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fac740664d0 0x7fac74068990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fac880d3010 0x7fac880cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 --2- 172.21.15.32:0/2082916858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880d15d0 0x7fac8806bcc0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 >> 172.21.15.32:0/2082916858 conn(0x7fac8809bc50 msgr2=0x7fac880d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:21.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 shutdown_connections 2023-10-16T08:19:21.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.096+0000 7fac8e446640 1 -- 172.21.15.32:0/2082916858 wait complete. 2023-10-16T08:19:21.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:19:21.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 Processor -- start 2023-10-16T08:19:21.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 -- start start 2023-10-16T08:19:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f986415ca70 0x7f98640d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98640d3550 0x7f98640cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640c65f0 con 0x7f98640cf5d0 2023-10-16T08:19:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98640ce820 con 0x7f986415ca70 2023-10-16T08:19:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986a380640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98640cfa40 con 0x7f98640d3550 2023-10-16T08:19:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986891d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98640cf5d0 0x7f98640d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34898/0 (socket says 172.21.15.32:34898) 2023-10-16T08:19:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f986891d640 1 -- 172.21.15.32:0/1206619308 learned_addr learned my addr 172.21.15.32:0/1206619308 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f9863fff640 1 --2- 172.21.15.32:0/1206619308 >> v2:172.21.15.32:3300/0 conn(0x7f986415ca70 0x7f98640d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f98637fe640 1 --2- 172.21.15.32:0/1206619308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98640d3550 0x7f98640cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f9862ffd640 1 -- 172.21.15.32:0/1206619308 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035475593 0 0) 0x7f98640c65f0 con 0x7f98640cf5d0 2023-10-16T08:19:21.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f9863fff640 1 -- 172.21.15.32:0/1206619308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98640d3550 msgr2=0x7f98640cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:21.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f9863fff640 1 --2- 172.21.15.32:0/1206619308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98640d3550 0x7f98640cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f9863fff640 1 -- 172.21.15.32:0/1206619308 >> v1:172.21.15.32:6789/0 conn(0x7f98640cf5d0 legacy=0x7f98640d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.260+0000 7f9863fff640 1 -- 172.21.15.32:0/1206619308 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98640cdf80 con 0x7f986415ca70 2023-10-16T08:19:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f9863fff640 1 --2- 172.21.15.32:0/1206619308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986415ca70 0x7f98640d3010 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f98540097d0 tx=0x7f985401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975ab83aac81d8da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f9862ffd640 1 -- 172.21.15.32:0/1206619308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540a7070 con 0x7f986415ca70 2023-10-16T08:19:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f9862ffd640 1 -- 172.21.15.32:0/1206619308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540ab430 con 0x7f986415ca70 2023-10-16T08:19:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f9862ffd640 1 -- 172.21.15.32:0/1206619308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540ab7c0 con 0x7f986415ca70 2023-10-16T08:19:21.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 -- 172.21.15.32:0/1206619308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986415ca70 msgr2=0x7f98640d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 --2- 172.21.15.32:0/1206619308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986415ca70 0x7f98640d3010 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f98540097d0 tx=0x7f985401c430 comp rx=0 tx=0).stop 2023-10-16T08:19:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 -- 172.21.15.32:0/1206619308 shutdown_connections 2023-10-16T08:19:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 --2- 172.21.15.32:0/1206619308 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f98640d3550 0x7f98640cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 --2- 172.21.15.32:0/1206619308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986415ca70 0x7f98640d3010 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 -- 172.21.15.32:0/1206619308 >> 172.21.15.32:0/1206619308 conn(0x7f986409bc50 msgr2=0x7f98640d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 -- 172.21.15.32:0/1206619308 shutdown_connections 2023-10-16T08:19:21.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.264+0000 7f986a380640 1 -- 172.21.15.32:0/1206619308 wait complete. 2023-10-16T08:19:21.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 Processor -- start 2023-10-16T08:19:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 -- start start 2023-10-16T08:19:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98640cf5d0 0x7f98640cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 0x7f9864153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f986415ca70 0x7f9864151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986414e620 con 0x7f98640d3550 2023-10-16T08:19:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98641537d0 con 0x7f98640cf5d0 2023-10-16T08:19:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9864156610 con 0x7f986415ca70 2023-10-16T08:19:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 0x7f9864153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 0x7f9864153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48868/0 (socket says 172.21.15.32:48868) 2023-10-16T08:19:21.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 -- 172.21.15.32:0/3123420699 learned_addr learned my addr 172.21.15.32:0/3123420699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f9863fff640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98640cf5d0 0x7f98640cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986891d640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f986415ca70 0x7f9864151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 -- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f986415ca70 msgr2=0x7f9864151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f986415ca70 0x7f9864151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 -- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98640cf5d0 msgr2=0x7f98640cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98640cf5d0 0x7f98640cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 -- 172.21.15.32:0/3123420699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98540093b0 con 0x7f98640d3550 2023-10-16T08:19:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986891d640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f986415ca70 0x7f9864151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f98637fe640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 0x7f9864153290 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f985800d830 tx=0x7f985800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800b660 con 0x7f98640d3550 2023-10-16T08:19:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985800b7c0 con 0x7f98640d3550 2023-10-16T08:19:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800bb30 con 0x7f98640d3550 2023-10-16T08:19:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9864157190 con 0x7f98640d3550 2023-10-16T08:19:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.268+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9864156b40 con 0x7f98640d3550 2023-10-16T08:19:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.272+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98640cf400 con 0x7f98640d3550 2023-10-16T08:19:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.272+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f985801e460 con 0x7f98640d3550 2023-10-16T08:19:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.272+0000 7f9860ff9640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98500664a0 0x7f9850068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.272+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f9858084670 con 0x7f98640d3550 2023-10-16T08:19:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.272+0000 7f9863fff640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98500664a0 0x7f9850068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.272+0000 7f9863fff640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98500664a0 0x7f9850068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f98540af000 tx=0x7f9854008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.276+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9858051980 con 0x7f98640d3550 2023-10-16T08:19:21.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.468+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9864074010 con 0x7f98640d3550 2023-10-16T08:19:21.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.468+0000 7f9860ff9640 1 -- 172.21.15.32:0/3123420699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f9858054fa0 con 0x7f98640d3550 2023-10-16T08:19:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:21.472 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:18:49.648104+0000","modified":"2023-10-16T08:18:52.672446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6834/3873719355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3873719355},{"type":"v1","addr":"172.21.15.32:6834","nonce":3873719355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T08:19:21.472 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2023-10-16T08:19:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.468+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98500664a0 msgr2=0x7f9850068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.468+0000 7f986a380640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98500664a0 0x7f9850068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f98540af000 tx=0x7f9854008040 comp rx=0 tx=0).stop 2023-10-16T08:19:21.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.468+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 msgr2=0x7f9864153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.468+0000 7f986a380640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 0x7f9864153290 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f985800d830 tx=0x7f985800dcf0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 shutdown_connections 2023-10-16T08:19:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f98500664a0 0x7f9850068960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f986415ca70 0x7f9864151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98640d3550 0x7f9864153290 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 --2- 172.21.15.32:0/3123420699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98640cf5d0 0x7f98640cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 >> 172.21.15.32:0/3123420699 conn(0x7f986409bc50 msgr2=0x7f986406ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 shutdown_connections 2023-10-16T08:19:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.472+0000 7f986a380640 1 -- 172.21.15.32:0/3123420699 wait complete. 2023-10-16T08:19:21.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 Processor -- start 2023-10-16T08:19:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 -- start start 2023-10-16T08:19:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fde5415cb30 0x7fde541608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde54065a80 0x7fde54160e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde540ccbd0 con 0x7fde5415cf10 2023-10-16T08:19:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5406b180 con 0x7fde54065a80 2023-10-16T08:19:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde58ad6640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5406b460 con 0x7fde5415cb30 2023-10-16T08:19:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde54065a80 0x7fde54160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde54065a80 0x7fde54160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48884/0 (socket says 172.21.15.32:48884) 2023-10-16T08:19:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde4bfff640 1 -- 172.21.15.32:0/1507439650 learned_addr learned my addr 172.21.15.32:0/1507439650 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.632+0000 7fde5259c640 1 --2- 172.21.15.32:0/1507439650 >> v2:172.21.15.186:3300/0 conn(0x7fde5415cb30 0x7fde541608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde4bfff640 1 -- 172.21.15.32:0/1507439650 >> v2:172.21.15.186:3300/0 conn(0x7fde5415cb30 msgr2=0x7fde541608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde4bfff640 1 --2- 172.21.15.32:0/1507439650 >> v2:172.21.15.186:3300/0 conn(0x7fde5415cb30 0x7fde541608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde4bfff640 1 -- 172.21.15.32:0/1507439650 >> v1:172.21.15.186:6789/0 conn(0x7fde5415cf10 legacy=0x7fde54069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde4bfff640 1 -- 172.21.15.32:0/1507439650 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5406c6e0 con 0x7fde54065a80 2023-10-16T08:19:21.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde4bfff640 1 --2- 172.21.15.32:0/1507439650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde54065a80 0x7fde54160e30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fde54072640 tx=0x7fde3c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdfa3ca09798be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde51d9b640 1 -- 172.21.15.32:0/1507439650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0ab670 con 0x7fde54065a80 2023-10-16T08:19:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde51d9b640 1 -- 172.21.15.32:0/1507439650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c0ab7d0 con 0x7fde54065a80 2023-10-16T08:19:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde51d9b640 1 -- 172.21.15.32:0/1507439650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0abb60 con 0x7fde54065a80 2023-10-16T08:19:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 -- 172.21.15.32:0/1507439650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde54065a80 msgr2=0x7fde54160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1507439650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde54065a80 0x7fde54160e30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fde54072640 tx=0x7fde3c01c4d0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 -- 172.21.15.32:0/1507439650 shutdown_connections 2023-10-16T08:19:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1507439650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde54065a80 0x7fde54160e30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1507439650 >> v2:172.21.15.186:3300/0 conn(0x7fde5415cb30 0x7fde541608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 -- 172.21.15.32:0/1507439650 >> 172.21.15.32:0/1507439650 conn(0x7fde540b0880 msgr2=0x7fde540b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 -- 172.21.15.32:0/1507439650 shutdown_connections 2023-10-16T08:19:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.636+0000 7fde58ad6640 1 -- 172.21.15.32:0/1507439650 wait complete. 2023-10-16T08:19:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 Processor -- start 2023-10-16T08:19:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 -- start start 2023-10-16T08:19:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde54065a80 0x7fde540c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde5415cb30 0x7fde5415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 0x7fde54159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5415af40 con 0x7fde5415cf10 2023-10-16T08:19:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5415b220 con 0x7fde54065a80 2023-10-16T08:19:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde58ad6640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5415b500 con 0x7fde5415cb30 2023-10-16T08:19:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 0x7fde54159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 0x7fde54159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48900/0 (socket says 172.21.15.32:48900) 2023-10-16T08:19:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 -- 172.21.15.32:0/1220618762 learned_addr learned my addr 172.21.15.32:0/1220618762 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde4bfff640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde5415cb30 0x7fde5415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde5259c640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde54065a80 0x7fde540c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 -- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde5415cb30 msgr2=0x7fde5415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde5415cb30 0x7fde5415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 -- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde54065a80 msgr2=0x7fde540c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde54065a80 0x7fde540c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 -- 172.21.15.32:0/1220618762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0094d0 con 0x7fde5415cf10 2023-10-16T08:19:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.640+0000 7fde52d9d640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 0x7fde54159ee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c730 tx=0x7fde4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c003b80 con 0x7fde5415cf10 2023-10-16T08:19:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c003ce0 con 0x7fde5415cf10 2023-10-16T08:19:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5415b7e0 con 0x7fde5415cf10 2023-10-16T08:19:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00aa30 con 0x7fde5415cf10 2023-10-16T08:19:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5415a420 con 0x7fde5415cf10 2023-10-16T08:19:21.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde540b0e70 con 0x7fde5415cf10 2023-10-16T08:19:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fde4c00ab90 con 0x7fde5415cf10 2023-10-16T08:19:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde4affd640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fde340664a0 0x7fde34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7fde4c083c60 con 0x7fde5415cf10 2023-10-16T08:19:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde5259c640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fde340664a0 0x7fde34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.644+0000 7fde5259c640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fde340664a0 0x7fde34068960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fde540b0290 tx=0x7fde44009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.648+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fde4c051020 con 0x7fde5415cf10 2023-10-16T08:19:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.808+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde54074010 con 0x7fde5415cf10 2023-10-16T08:19:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde4affd640 1 -- 172.21.15.32:0/1220618762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fde4c054640 con 0x7fde5415cf10 2023-10-16T08:19:21.814 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:21.814 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:18:51.254671+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"testcephfs1_metadata","create_time":"2023-10-16T08:18:48.142496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"testcephfs1_data","create_time":"2023-10-16T08:18:48.637534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fde340664a0 msgr2=0x7fde34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fde340664a0 0x7fde34068960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fde540b0290 tx=0x7fde44009190 comp rx=0 tx=0).stop 2023-10-16T08:19:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 msgr2=0x7fde54159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 0x7fde54159ee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c730 tx=0x7fde4c00cbf0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 shutdown_connections 2023-10-16T08:19:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde5415cf10 0x7fde54159ee0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fde340664a0 0x7fde34068960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fde5415cb30 0x7fde5415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 --2- 172.21.15.32:0/1220618762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde54065a80 0x7fde540c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 >> 172.21.15.32:0/1220618762 conn(0x7fde540b0880 msgr2=0x7fde540b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 shutdown_connections 2023-10-16T08:19:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.812+0000 7fde58ad6640 1 -- 172.21.15.32:0/1220618762 wait complete. 2023-10-16T08:19:21.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail testcephfs1 2023-10-16T08:19:21.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 Processor -- start 2023-10-16T08:19:21.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 -- start start 2023-10-16T08:19:21.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 0x7f447406af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f44740d1490 0x7f44740b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740d3510 con 0x7f447415cd60 2023-10-16T08:19:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740cd2b0 con 0x7f44740d1490 2023-10-16T08:19:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.976+0000 7f447ac2f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740d2f50 con 0x7f447415c980 2023-10-16T08:19:21.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 0x7f447406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 0x7f447406af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:44670/0 (socket says 172.21.15.32:44670) 2023-10-16T08:19:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 -- 172.21.15.32:0/2746736388 learned_addr learned my addr 172.21.15.32:0/2746736388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f447a42e640 1 --2- 172.21.15.32:0/2746736388 >> v2:172.21.15.186:3300/0 conn(0x7f44740d1490 0x7f44740b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 -- 172.21.15.32:0/2746736388 >> v2:172.21.15.186:3300/0 conn(0x7f44740d1490 msgr2=0x7f44740b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 --2- 172.21.15.32:0/2746736388 >> v2:172.21.15.186:3300/0 conn(0x7f44740d1490 0x7f44740b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 -- 172.21.15.32:0/2746736388 >> v1:172.21.15.186:6789/0 conn(0x7f447415cd60 legacy=0x7f44740d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 -- 172.21.15.32:0/2746736388 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740d3230 con 0x7f447415c980 2023-10-16T08:19:21.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4478c2b640 1 -- 172.21.15.32:0/2746736388 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42323974 0 0) 0x7f4464001120 con 0x7f447415cd60 2023-10-16T08:19:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.980+0000 7f4479c2d640 1 --2- 172.21.15.32:0/2746736388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 0x7f447406af90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4470009870 tx=0x7f447001c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824f4652bbe8dc8e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f4478c2b640 1 -- 172.21.15.32:0/2746736388 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700a7070 con 0x7f447415c980 2023-10-16T08:19:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f4478c2b640 1 -- 172.21.15.32:0/2746736388 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700ab430 con 0x7f447415c980 2023-10-16T08:19:21.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f4478c2b640 1 -- 172.21.15.32:0/2746736388 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700ab7c0 con 0x7f447415c980 2023-10-16T08:19:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- 172.21.15.32:0/2746736388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 msgr2=0x7f447406af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 --2- 172.21.15.32:0/2746736388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 0x7f447406af90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4470009870 tx=0x7f447001c520 comp rx=0 tx=0).stop 2023-10-16T08:19:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- 172.21.15.32:0/2746736388 shutdown_connections 2023-10-16T08:19:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 --2- 172.21.15.32:0/2746736388 >> v2:172.21.15.186:3300/0 conn(0x7f44740d1490 0x7f44740b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 --2- 172.21.15.32:0/2746736388 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f447415c980 0x7f447406af90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- 172.21.15.32:0/2746736388 >> 172.21.15.32:0/2746736388 conn(0x7f447409ba80 msgr2=0x7f44740d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- 172.21.15.32:0/2746736388 shutdown_connections 2023-10-16T08:19:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- 172.21.15.32:0/2746736388 wait complete. 2023-10-16T08:19:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 Processor -- start 2023-10-16T08:19:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- start start 2023-10-16T08:19:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44740d1490 0x7f4474141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f447415c980 0x7f4474155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 0x7f4474159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474154850 con 0x7f447415cd60 2023-10-16T08:19:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44741543c0 con 0x7f447415c980 2023-10-16T08:19:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447ac2f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44741422d0 con 0x7f44740d1490 2023-10-16T08:19:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447a42e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 0x7f4474159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447a42e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 0x7f4474159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48904/0 (socket says 172.21.15.32:48904) 2023-10-16T08:19:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.984+0000 7f447a42e640 1 -- 172.21.15.32:0/4044803881 learned_addr learned my addr 172.21.15.32:0/4044803881 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f4479c2d640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44740d1490 0x7f4474141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447942c640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f447415c980 0x7f4474155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447a42e640 1 -- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44740d1490 msgr2=0x7f4474141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447a42e640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44740d1490 0x7f4474141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447a42e640 1 -- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f447415c980 msgr2=0x7f4474155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447a42e640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f447415c980 0x7f4474155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447a42e640 1 -- 172.21.15.32:0/4044803881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009480 con 0x7f447415cd60 2023-10-16T08:19:21.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f4479c2d640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44740d1490 0x7f4474141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447942c640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f447415c980 0x7f4474155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447a42e640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 0x7f4474159ba0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f446c00efc0 tx=0x7f446c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c014480 con 0x7f447415cd60 2023-10-16T08:19:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00f040 con 0x7f447415cd60 2023-10-16T08:19:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c014820 con 0x7f447415cd60 2023-10-16T08:19:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44741537b0 con 0x7f447415cd60 2023-10-16T08:19:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4474153d70 con 0x7f447415cd60 2023-10-16T08:19:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4474154570 con 0x7f447415cd60 2023-10-16T08:19:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.988+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f446c014980 con 0x7f447415cd60 2023-10-16T08:19:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.992+0000 7f446a7fc640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44580664a0 0x7f4458068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.992+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6049+0+0 (secure 0 0 0) 0x7f446c084bf0 con 0x7f447415cd60 2023-10-16T08:19:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.992+0000 7f4479c2d640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44580664a0 0x7f4458068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.992+0000 7f4479c2d640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44580664a0 0x7f4458068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f44700af000 tx=0x7f44700a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:21.996+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4474154570 con 0x7f447415cd60 2023-10-16T08:19:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.184+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "testcephfs1"} v 0) v1 -- 0x7f4474154570 con 0x7f447415cd60 2023-10-16T08:19:22.188 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-10-16T08:19:22.184+0000 7efeb4e3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T08:19:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f446a7fc640 1 -- 172.21.15.32:0/4044803881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "testcephfs1"}]=0 testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4474154570 con 0x7f447415cd60 2023-10-16T08:19:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:testcephfs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T08:19:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44580664a0 msgr2=0x7f4458068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44580664a0 0x7f4458068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f44700af000 tx=0x7f44700a7000 comp rx=0 tx=0).stop 2023-10-16T08:19:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 msgr2=0x7f4474159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 0x7f4474159ba0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f446c00efc0 tx=0x7f446c00c490 comp rx=0 tx=0).stop 2023-10-16T08:19:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 shutdown_connections 2023-10-16T08:19:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447415cd60 0x7f4474159ba0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f447415c980 0x7f4474155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f44580664a0 0x7f4458068960 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 --2- 172.21.15.32:0/4044803881 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f44740d1490 0x7f4474141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.188+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 >> 172.21.15.32:0/4044803881 conn(0x7f447409ba80 msgr2=0x7f44740b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.192+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 shutdown_connections 2023-10-16T08:19:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.192+0000 7f447ac2f640 1 -- 172.21.15.32:0/4044803881 wait complete. 2023-10-16T08:19:22.199 INFO:tasks.ceph.mds.c.smithi032.stderr: -2701> 2023-10-16T08:18:52.236+0000 7fb5056e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:19:22.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm testcephfs1 --yes-i-really-mean-it 2023-10-16T08:19:22.247 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-10-16T08:19:22.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.360+0000 7f7e87c6b640 1 Processor -- start 2023-10-16T08:19:22.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.360+0000 7f7e87c6b640 1 -- start start 2023-10-16T08:19:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e87c6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e800d0210 0x7f7e800d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e87c6b640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f7e800b84e0 0x7f7e800b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e87c6b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e800d1670 con 0x7f7e8015cba0 2023-10-16T08:19:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e87c6b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e800d1460 con 0x7f7e800d0210 2023-10-16T08:19:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e87c6b640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e800d1950 con 0x7f7e800b84e0 2023-10-16T08:19:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85206640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e800d0210 0x7f7e800d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85a07640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e8015cba0 0x7f7e800cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34926/0 (socket says 172.21.15.32:34926) 2023-10-16T08:19:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85a07640 1 -- 172.21.15.32:0/2565741636 learned_addr learned my addr 172.21.15.32:0/2565741636 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e86208640 1 --2- 172.21.15.32:0/2565741636 >> v2:172.21.15.186:3300/0 conn(0x7f7e800b84e0 0x7f7e800b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85206640 1 -- 172.21.15.32:0/2565741636 >> v2:172.21.15.186:3300/0 conn(0x7f7e800b84e0 msgr2=0x7f7e800b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85206640 1 --2- 172.21.15.32:0/2565741636 >> v2:172.21.15.186:3300/0 conn(0x7f7e800b84e0 0x7f7e800b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85206640 1 -- 172.21.15.32:0/2565741636 >> v1:172.21.15.32:6789/0 conn(0x7f7e8015cba0 legacy=0x7f7e800cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e85206640 1 -- 172.21.15.32:0/2565741636 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e800d1c30 con 0x7f7e800d0210 2023-10-16T08:19:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e84a05640 1 -- 172.21.15.32:0/2565741636 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189338132 0 0) 0x7f7e800d1670 con 0x7f7e8015cba0 2023-10-16T08:19:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.364+0000 7f7e86208640 1 --2- 172.21.15.32:0/2565741636 >> v2:172.21.15.186:3300/0 conn(0x7f7e800b84e0 0x7f7e800b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e85206640 1 --2- 172.21.15.32:0/2565741636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e800d0210 0x7f7e800d0f20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7e800ce450 tx=0x7f7e7001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffbd0f51ab869ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e84a05640 1 -- 172.21.15.32:0/2565741636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700ac630 con 0x7f7e800d0210 2023-10-16T08:19:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e84a05640 1 -- 172.21.15.32:0/2565741636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e700ac790 con 0x7f7e800d0210 2023-10-16T08:19:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e84a05640 1 -- 172.21.15.32:0/2565741636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700acb20 con 0x7f7e800d0210 2023-10-16T08:19:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2565741636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e800d0210 msgr2=0x7f7e800d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2565741636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e800d0210 0x7f7e800d0f20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7e800ce450 tx=0x7f7e7001c3d0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2565741636 shutdown_connections 2023-10-16T08:19:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2565741636 >> v2:172.21.15.186:3300/0 conn(0x7f7e800b84e0 0x7f7e800b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2565741636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e800d0210 0x7f7e800d0f20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2565741636 >> 172.21.15.32:0/2565741636 conn(0x7f7e800719c0 msgr2=0x7f7e80071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2565741636 shutdown_connections 2023-10-16T08:19:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2565741636 wait complete. 2023-10-16T08:19:22.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 Processor -- start 2023-10-16T08:19:22.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.368+0000 7f7e87c6b640 1 -- start start 2023-10-16T08:19:22.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e800b84e0 0x7f7e80150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:22.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e800d0210 0x7f7e800c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:22.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 0x7f7e80156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800c6640 con 0x7f7e8015cba0 2023-10-16T08:19:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800c6ee0 con 0x7f7e800b84e0 2023-10-16T08:19:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80151500 con 0x7f7e800d0210 2023-10-16T08:19:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 0x7f7e80156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 0x7f7e80156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48948/0 (socket says 172.21.15.32:48948) 2023-10-16T08:19:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 -- 172.21.15.32:0/2587528459 learned_addr learned my addr 172.21.15.32:0/2587528459 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e85206640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e800d0210 0x7f7e800c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e85a07640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e800b84e0 0x7f7e80150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 -- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e800d0210 msgr2=0x7f7e800c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:22.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e800d0210 0x7f7e800c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 -- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e800b84e0 msgr2=0x7f7e80150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e800b84e0 0x7f7e80150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 -- 172.21.15.32:0/2587528459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009420 con 0x7f7e8015cba0 2023-10-16T08:19:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e85a07640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e800b84e0 0x7f7e80150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:22.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e86208640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 0x7f7e80156200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7e78007ea0 tx=0x7f7e78007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78015600 con 0x7f7e8015cba0 2023-10-16T08:19:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7800f040 con 0x7f7e8015cba0 2023-10-16T08:19:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e80156c80 con 0x7f7e8015cba0 2023-10-16T08:19:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e780159a0 con 0x7f7e8015cba0 2023-10-16T08:19:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e800c6b70 con 0x7f7e8015cba0 2023-10-16T08:19:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e800c7050 con 0x7f7e8015cba0 2023-10-16T08:19:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.372+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7e7801e3b0 con 0x7f7e8015cba0 2023-10-16T08:19:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.376+0000 7f7e767fc640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e640664d0 0x7f7e64068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.376+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6135+0+0 (secure 0 0 0) 0x7f7e78084290 con 0x7f7e8015cba0 2023-10-16T08:19:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.376+0000 7f7e85a07640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e640664d0 0x7f7e64068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.376+0000 7f7e85a07640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e640664d0 0x7f7e64068990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c000f70 tx=0x7f7e7c007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.380+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7e780516a0 con 0x7f7e8015cba0 2023-10-16T08:19:22.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:22.568+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7e8004e170 con 0x7f7e8015cba0 2023-10-16T08:19:23.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.188+0000 7f7e767fc640 1 -- 172.21.15.32:0/2587528459 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "testcephfs1", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7e78054cc0 con 0x7f7e8015cba0 2023-10-16T08:19:23.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e640664d0 msgr2=0x7f7e64068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:23.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e640664d0 0x7f7e64068990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c000f70 tx=0x7f7e7c007cb0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 msgr2=0x7f7e80156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 0x7f7e80156200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7e78007ea0 tx=0x7f7e78007f90 comp rx=0 tx=0).stop 2023-10-16T08:19:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 shutdown_connections 2023-10-16T08:19:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8015cba0 0x7f7e80156200 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f7e640664d0 0x7f7e64068990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f7e800d0210 0x7f7e800c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 --2- 172.21.15.32:0/2587528459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e800b84e0 0x7f7e80150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 >> 172.21.15.32:0/2587528459 conn(0x7f7e800719c0 msgr2=0x7f7e80071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 shutdown_connections 2023-10-16T08:19:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.192+0000 7f7e87c6b640 1 -- 172.21.15.32:0/2587528459 wait complete. 2023-10-16T08:19:23.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_metadata testcephfs1_metadata --yes-i-really-really-mean-it 2023-10-16T08:19:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc4114ac640 1 Processor -- start 2023-10-16T08:19:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc4114ac640 1 -- start start 2023-10-16T08:19:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc4114ac640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc40c15de90 0x7fc40c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc4114ac640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c070ce0 con 0x7fc40c15dab0 2023-10-16T08:19:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc4114ac640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c0700d0 con 0x7fc40c0b3e50 2023-10-16T08:19:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc4114ac640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c0703b0 con 0x7fc40c15de90 2023-10-16T08:19:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc40c15de90 0x7fc40c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc40c15dab0 0x7fc40c15ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57044/0 (socket says 172.21.15.32:57044) 2023-10-16T08:19:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40bfff640 1 -- 172.21.15.32:0/2127622703 learned_addr learned my addr 172.21.15.32:0/2127622703 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40b7fe640 1 -- 172.21.15.32:0/2127622703 >> v1:172.21.15.186:6789/0 conn(0x7fc40c0b3e50 legacy=0x7fc40c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40b7fe640 1 -- 172.21.15.32:0/2127622703 >> v1:172.21.15.32:6789/0 conn(0x7fc40c15dab0 legacy=0x7fc40c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40affd640 1 -- 172.21.15.32:0/2127622703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220037053 0 0) 0x7fc40c070ce0 con 0x7fc40c15dab0 2023-10-16T08:19:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.408+0000 7fc40b7fe640 1 -- 172.21.15.32:0/2127622703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c070690 con 0x7fc40c15de90 2023-10-16T08:19:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc40b7fe640 1 --2- 172.21.15.32:0/2127622703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c15de90 0x7fc40c0cab90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc4000097c0 tx=0x7fc40001c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e5ce58c305983a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc40affd640 1 -- 172.21.15.32:0/2127622703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000ab600 con 0x7fc40c15de90 2023-10-16T08:19:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc40affd640 1 -- 172.21.15.32:0/2127622703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4000a7050 con 0x7fc40c15de90 2023-10-16T08:19:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc40affd640 1 -- 172.21.15.32:0/2127622703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000ab980 con 0x7fc40c15de90 2023-10-16T08:19:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 -- 172.21.15.32:0/2127622703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c15de90 msgr2=0x7fc40c0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 --2- 172.21.15.32:0/2127622703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c15de90 0x7fc40c0cab90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc4000097c0 tx=0x7fc40001c2f0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 -- 172.21.15.32:0/2127622703 shutdown_connections 2023-10-16T08:19:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 --2- 172.21.15.32:0/2127622703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c15de90 0x7fc40c0cab90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 -- 172.21.15.32:0/2127622703 >> 172.21.15.32:0/2127622703 conn(0x7fc40c06a6e0 msgr2=0x7fc40c0afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 -- 172.21.15.32:0/2127622703 shutdown_connections 2023-10-16T08:19:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 -- 172.21.15.32:0/2127622703 wait complete. 2023-10-16T08:19:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 Processor -- start 2023-10-16T08:19:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.412+0000 7fc4114ac640 1 -- start start 2023-10-16T08:19:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 0x7fc40c0c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc40c15dab0 0x7fc40c15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc40c15de90 0x7fc40c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c1550d0 con 0x7fc40c0b3e50 2023-10-16T08:19:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c154bb0 con 0x7fc40c15dab0 2023-10-16T08:19:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c1545f0 con 0x7fc40c15de90 2023-10-16T08:19:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 0x7fc40c0c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 0x7fc40c0c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35032/0 (socket says 172.21.15.32:35032) 2023-10-16T08:19:23.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 -- 172.21.15.32:0/1070674895 learned_addr learned my addr 172.21.15.32:0/1070674895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:23.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40b7fe640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc40c15dab0 0x7fc40c15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:23.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc410cab640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc40c15de90 0x7fc40c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:23.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 -- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc40c15de90 msgr2=0x7fc40c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:23.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc40c15de90 0x7fc40c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 -- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc40c15dab0 msgr2=0x7fc40c15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc40c15dab0 0x7fc40c15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 -- 172.21.15.32:0/1070674895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc400009420 con 0x7fc40c0b3e50 2023-10-16T08:19:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc410cab640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc40c15de90 0x7fc40c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc40bfff640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 0x7fc40c0c8360 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0011f0 tx=0x7fc3fc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0074e0 con 0x7fc40c0b3e50 2023-10-16T08:19:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc007640 con 0x7fc40c0b3e50 2023-10-16T08:19:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc004a10 con 0x7fc40c0b3e50 2023-10-16T08:19:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c1548d0 con 0x7fc40c0b3e50 2023-10-16T08:19:23.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c15b880 con 0x7fc40c0b3e50 2023-10-16T08:19:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.416+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c04e9c0 con 0x7fc40c0b3e50 2023-10-16T08:19:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.420+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc3fc004b70 con 0x7fc40c0b3e50 2023-10-16T08:19:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.420+0000 7fc408ff9640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc3f80664a0 0x7fc3f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.420+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6135+0+0 (secure 0 0 0) 0x7fc3fc083270 con 0x7fc40c0b3e50 2023-10-16T08:19:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.420+0000 7fc40b7fe640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc3f80664a0 0x7fc3f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.420+0000 7fc40b7fe640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc3f80664a0 0x7fc3f8068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc4000ad000 tx=0x7fc4000a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.424+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc3fc0505e0 con 0x7fc40c0b3e50 2023-10-16T08:19:23.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:23.580+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc40c0b0b90 con 0x7fc40c0b3e50 2023-10-16T08:19:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.192+0000 7fc408ff9640 1 -- 172.21.15.32:0/1070674895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_metadata", "pool2": "testcephfs1_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_metadata' removed v56) v1 ==== 196+0+0 (secure 0 0 0) 0x7fc3fc0180e0 con 0x7fc40c0b3e50 2023-10-16T08:19:24.224 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_metadata' removed 2023-10-16T08:19:24.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc3f80664a0 msgr2=0x7fc3f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc3f80664a0 0x7fc3f8068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc4000ad000 tx=0x7fc4000a6000 comp rx=0 tx=0).stop 2023-10-16T08:19:24.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 msgr2=0x7fc40c0c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 0x7fc40c0c8360 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0011f0 tx=0x7fc3fc00c9b0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 shutdown_connections 2023-10-16T08:19:24.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fc3f80664a0 0x7fc3f8068960 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fc40c15de90 0x7fc40c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc40c15dab0 0x7fc40c15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 --2- 172.21.15.32:0/1070674895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc40c0b3e50 0x7fc40c0c8360 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 >> 172.21.15.32:0/1070674895 conn(0x7fc40c06a6e0 msgr2=0x7fc40c155360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:24.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.196+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 shutdown_connections 2023-10-16T08:19:24.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.200+0000 7fc4114ac640 1 -- 172.21.15.32:0/1070674895 wait complete. 2023-10-16T08:19:24.228 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm testcephfs1_data testcephfs1_data --yes-i-really-really-mean-it 2023-10-16T08:19:24.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2de854640 1 Processor -- start 2023-10-16T08:19:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2de854640 1 -- start start 2023-10-16T08:19:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2de854640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2d80cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2de854640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80caf20 con 0x7fb2d81600e0 2023-10-16T08:19:24.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2de854640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80b3720 con 0x7fb2d80cd2b0 2023-10-16T08:19:24.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2de854640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80d02a0 con 0x7fb2d80cd740 2023-10-16T08:19:24.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb2d81600e0 0x7fb2d80cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57046/0 (socket says 172.21.15.32:57046) 2023-10-16T08:19:24.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d7fff640 1 -- 172.21.15.32:0/2463194695 learned_addr learned my addr 172.21.15.32:0/2463194695 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2d80cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818039649 0 0) 0x7fb2d80caf20 con 0x7fb2d81600e0 2023-10-16T08:19:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c0003530 con 0x7fb2d81600e0 2023-10-16T08:19:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3325904304 0 0) 0x7fb2c0003530 con 0x7fb2d81600e0 2023-10-16T08:19:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 msgr2=0x7fb2d80cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2d80cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 >> v1:172.21.15.186:6789/0 conn(0x7fb2d80cd2b0 legacy=0x7fb2d80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80cb170 con 0x7fb2d81600e0 2023-10-16T08:19:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fb2c8001680 con 0x7fb2d81600e0 2023-10-16T08:19:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 >> v1:172.21.15.32:6789/0 conn(0x7fb2d81600e0 legacy=0x7fb2d80cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2c0003f30 0x7fb2c00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c0004d10 0x7fb2c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80d02a0 con 0x7fb2c0004d10 2023-10-16T08:19:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c000bb20 con 0x7fb2c0003f30 2023-10-16T08:19:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c000bce0 con 0x7fb2d80cd740 2023-10-16T08:19:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c0004d10 0x7fb2c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d7fff640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2c0003f30 0x7fb2c00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 -- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 msgr2=0x7fb2c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 -- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2c0003f30 msgr2=0x7fb2c00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2c0003f30 0x7fb2c00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2dcdf1640 1 -- 172.21.15.32:0/2463194695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0003530 con 0x7fb2c0004d10 2023-10-16T08:19:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.376+0000 7fb2d7fff640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d80cd740 0x7fb2c00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c0004d10 0x7fb2c0005190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009fc0 tx=0x7fb2d001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eddc0f046116b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a5c70 con 0x7fb2c0004d10 2023-10-16T08:19:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d000f040 con 0x7fb2c0004d10 2023-10-16T08:19:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/2463194695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00ad440 con 0x7fb2c0004d10 2023-10-16T08:19:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2de854640 1 -- 172.21.15.32:0/2463194695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c0004d10 msgr2=0x7fb2c0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2de854640 1 --2- 172.21.15.32:0/2463194695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c0004d10 0x7fb2c0005190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009fc0 tx=0x7fb2d001cc60 comp rx=0 tx=0).stop 2023-10-16T08:19:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2dcdf1640 1 -- 172.21.15.32:0/2463194695 reap_dead start 2023-10-16T08:19:24.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2de854640 1 -- 172.21.15.32:0/2463194695 shutdown_connections 2023-10-16T08:19:24.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2de854640 1 -- 172.21.15.32:0/2463194695 >> 172.21.15.32:0/2463194695 conn(0x7fb2d80d08d0 msgr2=0x7fb2d80d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:24.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2de854640 1 -- 172.21.15.32:0/2463194695 shutdown_connections 2023-10-16T08:19:24.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.380+0000 7fb2de854640 1 -- 172.21.15.32:0/2463194695 wait complete. 2023-10-16T08:19:24.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 Processor -- start 2023-10-16T08:19:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 -- start start 2023-10-16T08:19:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 0x7fb2d80b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80b4f50 0x7fb2d8149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d8149950 0x7fb2d80c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c4db0 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80ca550 con 0x7fb2d80b4f50 2023-10-16T08:19:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c95c0 con 0x7fb2d8149950 2023-10-16T08:19:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 0x7fb2d80b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 0x7fb2d80b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35052/0 (socket says 172.21.15.32:35052) 2023-10-16T08:19:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 -- 172.21.15.32:0/3936150822 learned_addr learned my addr 172.21.15.32:0/3936150822 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d8149950 0x7fb2d80c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80b4f50 0x7fb2d8149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 -- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d8149950 msgr2=0x7fb2d80c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d8149950 0x7fb2d80c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 -- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80b4f50 msgr2=0x7fb2d8149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80b4f50 0x7fb2d8149410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 -- 172.21.15.32:0/3936150822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80c4480 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2dcdf1640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d8149950 0x7fb2d80c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80b4f50 0x7fb2d8149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d7fff640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 0x7fb2d80b4a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000f70 tx=0x7fb2c8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800fc20 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c800fd80 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8015610 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d80c4760 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.384+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d80b53d0 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.388+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d804e170 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.388+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2c8015770 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.388+0000 7fb2d4ff9640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb2c40664a0 0x7fb2c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:24.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.388+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5687+0+0 (secure 0 0 0) 0x7fb2c8087320 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.388+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb2c40664a0 0x7fb2c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.388+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb2c40664a0 0x7fb2c4068960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb2d815e160 tx=0x7fb2cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.392+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2c80547b0 con 0x7fb2d80b5ad0 2023-10-16T08:19:24.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:24.552+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2d80cabe0 con 0x7fb2d80b5ad0 2023-10-16T08:19:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.200+0000 7fb2d4ff9640 1 -- 172.21.15.32:0/3936150822 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "testcephfs1_data", "pool2": "testcephfs1_data", "yes_i_really_really_mean_it": true}]=0 pool 'testcephfs1_data' removed v57) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb2c8057dd0 con 0x7fb2d80b5ad0 2023-10-16T08:19:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'testcephfs1_data' removed 2023-10-16T08:19:25.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb2c40664a0 msgr2=0x7fb2c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb2c40664a0 0x7fb2c4068960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb2d815e160 tx=0x7fb2cc009040 comp rx=0 tx=0).stop 2023-10-16T08:19:25.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 msgr2=0x7fb2d80b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 0x7fb2d80b4a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000f70 tx=0x7fb2c8011de0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 shutdown_connections 2023-10-16T08:19:25.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb2c40664a0 0x7fb2c4068960 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb2d8149950 0x7fb2d80c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2d80b4f50 0x7fb2d8149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 --2- 172.21.15.32:0/3936150822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d80b5ad0 0x7fb2d80b4a10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.204+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 >> 172.21.15.32:0/3936150822 conn(0x7fb2d80d08d0 msgr2=0x7fb2d80d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.208+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 shutdown_connections 2023-10-16T08:19:25.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.208+0000 7fb2de854640 1 -- 172.21.15.32:0/3936150822 wait complete. 2023-10-16T08:19:25.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T08:19:25.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 Processor -- start 2023-10-16T08:19:25.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 -- start start 2023-10-16T08:19:25.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f030c15cba0 0x7f030c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c0d0210 0x7f030c0b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030c0d1670 con 0x7f030c0b84e0 2023-10-16T08:19:25.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030c0d1460 con 0x7f030c15cba0 2023-10-16T08:19:25.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0312794640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0d1950 con 0x7f030c0d0210 2023-10-16T08:19:25.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f030c15cba0 0x7f030c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f030c15cba0 0x7f030c0d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35062/0 (socket says 172.21.15.32:35062) 2023-10-16T08:19:25.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 -- 172.21.15.32:0/2901423280 learned_addr learned my addr 172.21.15.32:0/2901423280 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:25.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0310d31640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c0d0210 0x7f030c0b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 -- 172.21.15.32:0/2901423280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c0d0210 msgr2=0x7f030c0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:25.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c0d0210 0x7f030c0b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 -- 172.21.15.32:0/2901423280 >> v1:172.21.15.186:6789/0 conn(0x7f030c0b84e0 legacy=0x7f030c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f030bfff640 1 -- 172.21.15.32:0/2901423280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c0d1c30 con 0x7f030c15cba0 2023-10-16T08:19:25.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.392+0000 7f0310d31640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c0d0210 0x7f030c0b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:25.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f030bfff640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c15cba0 0x7f030c0d0f20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8850d64ac5086867 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:25.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f030affd640 1 -- 172.21.15.32:0/2901423280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0a9400 con 0x7f030c15cba0 2023-10-16T08:19:25.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f030affd640 1 -- 172.21.15.32:0/2901423280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0a9560 con 0x7f030c15cba0 2023-10-16T08:19:25.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f030affd640 1 -- 172.21.15.32:0/2901423280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0a98f0 con 0x7f030c15cba0 2023-10-16T08:19:25.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 -- 172.21.15.32:0/2901423280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c15cba0 msgr2=0x7f030c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c15cba0 0x7f030c0d0f20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc01c540 comp rx=0 tx=0).stop 2023-10-16T08:19:25.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 -- 172.21.15.32:0/2901423280 shutdown_connections 2023-10-16T08:19:25.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c0d0210 0x7f030c0b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 --2- 172.21.15.32:0/2901423280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c15cba0 0x7f030c0d0f20 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 -- 172.21.15.32:0/2901423280 >> 172.21.15.32:0/2901423280 conn(0x7f030c0719c0 msgr2=0x7f030c071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:25.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 -- 172.21.15.32:0/2901423280 shutdown_connections 2023-10-16T08:19:25.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.396+0000 7f0312794640 1 -- 172.21.15.32:0/2901423280 wait complete. 2023-10-16T08:19:25.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 Processor -- start 2023-10-16T08:19:25.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 -- start start 2023-10-16T08:19:25.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 0x7f030c14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f030c0d0210 0x7f030c155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c15cba0 0x7f030c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c157df0 con 0x7f030c0b84e0 2023-10-16T08:19:25.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c157470 con 0x7f030c0d0210 2023-10-16T08:19:25.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0c8970 con 0x7f030c15cba0 2023-10-16T08:19:25.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 0x7f030c14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 0x7f030c14e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35076/0 (socket says 172.21.15.32:35076) 2023-10-16T08:19:25.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 -- 172.21.15.32:0/2237149981 learned_addr learned my addr 172.21.15.32:0/2237149981 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:25.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0310d31640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c15cba0 0x7f030c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030b7fe640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f030c0d0210 0x7f030c155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 -- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c15cba0 msgr2=0x7f030c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c15cba0 0x7f030c1555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 -- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f030c0d0210 msgr2=0x7f030c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f030c0d0210 0x7f030c155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 -- 172.21.15.32:0/2237149981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0094d0 con 0x7f030c0b84e0 2023-10-16T08:19:25.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0310d31640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c15cba0 0x7f030c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f030bfff640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 0x7f030c14e320 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f02fc01abf0 tx=0x7f02fc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0b18c0 con 0x7f030c0b84e0 2023-10-16T08:19:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0b1a40 con 0x7f030c0b84e0 2023-10-16T08:19:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c157730 con 0x7f030c0b84e0 2023-10-16T08:19:25.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0b1df0 con 0x7f030c0b84e0 2023-10-16T08:19:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.400+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c14e860 con 0x7f030c0b84e0 2023-10-16T08:19:25.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.404+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030c0665b0 con 0x7f030c0b84e0 2023-10-16T08:19:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.404+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f02fc0bc450 con 0x7f030c0b84e0 2023-10-16T08:19:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.404+0000 7f0308ff9640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f02f80664a0 0x7f02f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.404+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f02fc11e820 con 0x7f030c0b84e0 2023-10-16T08:19:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.404+0000 7f030b7fe640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f02f80664a0 0x7f02f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.404+0000 7f030b7fe640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f02f80664a0 0x7f02f8068960 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f030c1f8ac0 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.408+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f02fc0ec020 con 0x7f030c0b84e0 2023-10-16T08:19:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.568+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f030c04e170 con 0x7f030c0b84e0 2023-10-16T08:19:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0308ff9640 1 -- 172.21.15.32:0/2237149981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02fc0a6030 con 0x7f030c0b84e0 2023-10-16T08:19:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f02f80664a0 msgr2=0x7f02f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f02f80664a0 0x7f02f8068960 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f030c1f8ac0 tx=0x7f0300009040 comp rx=0 tx=0).stop 2023-10-16T08:19:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 msgr2=0x7f030c14e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 0x7f030c14e320 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f02fc01abf0 tx=0x7f02fc01ad80 comp rx=0 tx=0).stop 2023-10-16T08:19:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 shutdown_connections 2023-10-16T08:19:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f02f80664a0 0x7f02f8068960 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f030c15cba0 0x7f030c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f030c0d0210 0x7f030c155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 --2- 172.21.15.32:0/2237149981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0b84e0 0x7f030c14e320 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 >> 172.21.15.32:0/2237149981 conn(0x7f030c0719c0 msgr2=0x7f030c15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 shutdown_connections 2023-10-16T08:19:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.572+0000 7f0312794640 1 -- 172.21.15.32:0/2237149981 wait complete. 2023-10-16T08:19:25.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-10-16T08:19:25.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 Processor -- start 2023-10-16T08:19:25.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 -- start start 2023-10-16T08:19:25.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd12c159ba0 0x7fd12c0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd12c0adc10 0x7fd12c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0723d0 con 0x7fd12c159ba0 2023-10-16T08:19:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0717c0 con 0x7fd12c0adc10 2023-10-16T08:19:25.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.736+0000 7fd133668640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c071aa0 con 0x7fd12c1597c0 2023-10-16T08:19:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd12c159ba0 0x7fd12c0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd12c159ba0 0x7fd12c0cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35084/0 (socket says 172.21.15.32:35084) 2023-10-16T08:19:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 -- 172.21.15.32:0/1193995889 learned_addr learned my addr 172.21.15.32:0/1193995889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd131404640 1 --2- 172.21.15.32:0/1193995889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd131c05640 1 --2- 172.21.15.32:0/1193995889 >> v2:172.21.15.186:3300/0 conn(0x7fd12c0adc10 0x7fd12c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 -- 172.21.15.32:0/1193995889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 msgr2=0x7fd12c0ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 --2- 172.21.15.32:0/1193995889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c0ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 -- 172.21.15.32:0/1193995889 >> v2:172.21.15.186:3300/0 conn(0x7fd12c0adc10 msgr2=0x7fd12c0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 --2- 172.21.15.32:0/1193995889 >> v2:172.21.15.186:3300/0 conn(0x7fd12c0adc10 0x7fd12c0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 -- 172.21.15.32:0/1193995889 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c071d80 con 0x7fd12c159ba0 2023-10-16T08:19:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd130c03640 1 --2- 172.21.15.32:0/1193995889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c159ba0 0x7fd12c0cae10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06e330 tx=0x7fd11c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f671672147428f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd123fff640 1 -- 172.21.15.32:0/1193995889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0a7070 con 0x7fd12c159ba0 2023-10-16T08:19:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd123fff640 1 -- 172.21.15.32:0/1193995889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c0ab430 con 0x7fd12c159ba0 2023-10-16T08:19:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.740+0000 7fd123fff640 1 -- 172.21.15.32:0/1193995889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0ab7c0 con 0x7fd12c159ba0 2023-10-16T08:19:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- 172.21.15.32:0/1193995889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c159ba0 msgr2=0x7fd12c0cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- 172.21.15.32:0/1193995889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c159ba0 0x7fd12c0cae10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06e330 tx=0x7fd11c01c4d0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- 172.21.15.32:0/1193995889 shutdown_connections 2023-10-16T08:19:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- 172.21.15.32:0/1193995889 >> v2:172.21.15.186:3300/0 conn(0x7fd12c0adc10 0x7fd12c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- 172.21.15.32:0/1193995889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c159ba0 0x7fd12c0cae10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- 172.21.15.32:0/1193995889 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- 172.21.15.32:0/1193995889 >> 172.21.15.32:0/1193995889 conn(0x7fd12c06c190 msgr2=0x7fd12c06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:25.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- 172.21.15.32:0/1193995889 shutdown_connections 2023-10-16T08:19:25.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- 172.21.15.32:0/1193995889 wait complete. 2023-10-16T08:19:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 Processor -- start 2023-10-16T08:19:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- start start 2023-10-16T08:19:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 0x7fd12c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c159ba0 0x7fd12c14a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c157d40 con 0x7fd12c0adc10 2023-10-16T08:19:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c157130 con 0x7fd12c159ba0 2023-10-16T08:19:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd133668640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c157410 con 0x7fd12c1597c0 2023-10-16T08:19:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 0x7fd12c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 0x7fd12c149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35100/0 (socket says 172.21.15.32:35100) 2023-10-16T08:19:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 -- 172.21.15.32:0/1780082739 learned_addr learned my addr 172.21.15.32:0/1780082739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 -- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 msgr2=0x7fd12c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 -- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c159ba0 msgr2=0x7fd12c14a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c159ba0 0x7fd12c14a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.744+0000 7fd131404640 1 -- 172.21.15.32:0/1780082739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c0094d0 con 0x7fd12c0adc10 2023-10-16T08:19:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd131404640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 0x7fd12c149dc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd12800e930 tx=0x7fd12800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128009690 con 0x7fd12c0adc10 2023-10-16T08:19:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800ca00 con 0x7fd12c0adc10 2023-10-16T08:19:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12800cd70 con 0x7fd12c0adc10 2023-10-16T08:19:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c1576f0 con 0x7fd12c0adc10 2023-10-16T08:19:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c156dc0 con 0x7fd12c0adc10 2023-10-16T08:19:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c074010 con 0x7fd12c0adc10 2023-10-16T08:19:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd128013470 con 0x7fd12c0adc10 2023-10-16T08:19:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd121ffb640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1100664f0 0x7fd1100689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd128083e30 con 0x7fd12c0adc10 2023-10-16T08:19:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.748+0000 7fd130c03640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1100664f0 0x7fd1100689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.752+0000 7fd130c03640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1100664f0 0x7fd1100689b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0af000 tx=0x7fd11c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.752+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd1280515a0 con 0x7fd12c0adc10 2023-10-16T08:19:25.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:25.968+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fd12c0cd3f0 con 0x7fd12c0adc10 2023-10-16T08:19:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd121ffb640 1 -- 172.21.15.32:0/1780082739 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v193) v1 ==== 134+0+0 (secure 0 0 0) 0x7fd1280170c0 con 0x7fd12c0adc10 2023-10-16T08:19:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1100664f0 msgr2=0x7fd1100689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1100664f0 0x7fd1100689b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0af000 tx=0x7fd11c004740 comp rx=0 tx=0).stop 2023-10-16T08:19:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 msgr2=0x7fd12c149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 0x7fd12c149dc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd12800e930 tx=0x7fd12800edf0 comp rx=0 tx=0).stop 2023-10-16T08:19:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 shutdown_connections 2023-10-16T08:19:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd12c159ba0 0x7fd12c14a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd1100664f0 0x7fd1100689b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd12c1597c0 0x7fd12c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 --2- 172.21.15.32:0/1780082739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd12c0adc10 0x7fd12c149dc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 >> 172.21.15.32:0/1780082739 conn(0x7fd12c06c190 msgr2=0x7fd12c0cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 shutdown_connections 2023-10-16T08:19:26.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:26.204+0000 7fd133668640 1 -- 172.21.15.32:0/1780082739 wait complete. 2023-10-16T08:19:26.227 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-10-16T08:19:26.228 INFO:tasks.cephfs_test_runner: 2023-10-16T08:19:26.228 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T08:19:26.229 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T08:19:26.229 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T08:19:26.229 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T08:19:26.229 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-10-16T08:19:26.230 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-10-16T08:19:26.230 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-10-16T08:19:26.230 INFO:tasks.cephfs_test_runner: p.wait() 2023-10-16T08:19:26.230 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T08:19:26.231 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T08:19:26.231 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T08:19:26.231 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T08:19:26.231 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-10-16T08:19:26.232 INFO:tasks.cephfs_test_runner: 2023-10-16T08:19:26.232 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T08:19:26.232 INFO:tasks.cephfs_test_runner:Ran 2 tests in 130.768s 2023-10-16T08:19:26.232 INFO:tasks.cephfs_test_runner: 2023-10-16T08:19:26.233 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-16T08:19:26.233 INFO:tasks.cephfs_test_runner: 2023-10-16T08:19:26.233 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T08:19:26.233 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T08:19:26.234 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T08:19:26.234 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T08:19:26.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-10-16T08:19:26.234 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-10-16T08:19:26.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-10-16T08:19:26.235 INFO:tasks.cephfs_test_runner: p.wait() 2023-10-16T08:19:26.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T08:19:26.235 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T08:19:26.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T08:19:26.236 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T08:19:26.236 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-10-16T08:19:26.236 INFO:tasks.cephfs_test_runner: 2023-10-16T08:19:26.237 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T08:19:26.304 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0a17552bbd354a94856fe88fb057eb4a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_75071e9ab3c3bca3308a5f42b10f82d632c58444/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-16T08:19:26.307 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-16T08:19:26.318 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-16T08:19:26.332 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-16T08:19:26.333 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:26.333 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:26.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:19:26.356 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-10-16T08:19:26.356 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:26.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6538 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-16T08:19:26.409 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:19:26.410 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:26.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:19:26.420 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T08:19:26.420 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:19:26.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354672k,nr_inodes=4088668,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4464 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-16T08:19:26.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.472 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2023-10-16T08:19:26.524 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T08:19:26.524 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-16T08:19:26.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.525 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:19:26.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.593 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:19:26.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.680 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2023-10-16T08:19:26.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.899 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-10-16T08:19:26.909 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:19:26.909 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:19:26.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:19:26.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:26.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:19:27.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:19:27.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-10-16T08:19:27.275 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-16T08:19:27.287 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T08:19:27.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:19:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.432+0000 7f9bfd19d640 1 Processor -- start 2023-10-16T08:19:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.432+0000 7f9bfd19d640 1 -- start start 2023-10-16T08:19:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bfd19d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f9bf815cf00 0x7f9bf80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bfd19d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bf80d1410 0x7f9bf80b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bfd19d640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80ce340 con 0x7f9bf80cdee0 2023-10-16T08:19:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bfd19d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf815d2e0 con 0x7f9bf80d1410 2023-10-16T08:19:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bfd19d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80d1930 con 0x7f9bf815cf00 2023-10-16T08:19:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bf80d1410 0x7f9bf80b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bf80d1410 0x7f9bf80b6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35102/0 (socket says 172.21.15.32:35102) 2023-10-16T08:19:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 -- 172.21.15.32:0/2664942786 learned_addr learned my addr 172.21.15.32:0/2664942786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf6d9d640 1 --2- 172.21.15.32:0/2664942786 >> v2:172.21.15.186:3300/0 conn(0x7f9bf815cf00 0x7f9bf80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 -- 172.21.15.32:0/2664942786 >> v2:172.21.15.186:3300/0 conn(0x7f9bf815cf00 msgr2=0x7f9bf80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 --2- 172.21.15.32:0/2664942786 >> v2:172.21.15.186:3300/0 conn(0x7f9bf815cf00 0x7f9bf80cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 -- 172.21.15.32:0/2664942786 >> v1:172.21.15.186:6789/0 conn(0x7f9bf80cdee0 legacy=0x7f9bf80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 -- 172.21.15.32:0/2664942786 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80d1c10 con 0x7f9bf80d1410 2023-10-16T08:19:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf759e640 1 --2- 172.21.15.32:0/2664942786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80d1410 0x7f9bf80b6650 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806bff0 tx=0x7f9bf001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752f285c014a7f0a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.436+0000 7f9bf659c640 1 -- 172.21.15.32:0/2664942786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00a5800 con 0x7f9bf80d1410 2023-10-16T08:19:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bf659c640 1 -- 172.21.15.32:0/2664942786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf00a5960 con 0x7f9bf80d1410 2023-10-16T08:19:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bf659c640 1 -- 172.21.15.32:0/2664942786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00a5cf0 con 0x7f9bf80d1410 2023-10-16T08:19:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- 172.21.15.32:0/2664942786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80d1410 msgr2=0x7f9bf80b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/2664942786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80d1410 0x7f9bf80b6650 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806bff0 tx=0x7f9bf001c940 comp rx=0 tx=0).stop 2023-10-16T08:19:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- 172.21.15.32:0/2664942786 shutdown_connections 2023-10-16T08:19:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/2664942786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80d1410 0x7f9bf80b6650 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/2664942786 >> v2:172.21.15.186:3300/0 conn(0x7f9bf815cf00 0x7f9bf80cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- 172.21.15.32:0/2664942786 >> 172.21.15.32:0/2664942786 conn(0x7f9bf80b4470 msgr2=0x7f9bf80b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- 172.21.15.32:0/2664942786 shutdown_connections 2023-10-16T08:19:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- 172.21.15.32:0/2664942786 wait complete. 2023-10-16T08:19:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 Processor -- start 2023-10-16T08:19:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- start start 2023-10-16T08:19:27.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 0x7f9bf814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bf80d1410 0x7f9bf81f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bf815cf00 0x7f9bf81f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815bed0 con 0x7f9bf80cdee0 2023-10-16T08:19:27.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815a690 con 0x7f9bf80d1410 2023-10-16T08:19:27.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bfd19d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815a970 con 0x7f9bf815cf00 2023-10-16T08:19:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bf6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 0x7f9bf814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bf6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 0x7f9bf814f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35116/0 (socket says 172.21.15.32:35116) 2023-10-16T08:19:27.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.440+0000 7f9bf6d9d640 1 -- 172.21.15.32:0/4060475535 learned_addr learned my addr 172.21.15.32:0/4060475535 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9beffff640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bf80d1410 0x7f9bf81f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf759e640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bf815cf00 0x7f9bf81f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf6d9d640 1 -- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bf815cf00 msgr2=0x7f9bf81f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf6d9d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bf815cf00 0x7f9bf81f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf6d9d640 1 -- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bf80d1410 msgr2=0x7f9bf81f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf6d9d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bf80d1410 0x7f9bf81f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf6d9d640 1 -- 172.21.15.32:0/4060475535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009b30 con 0x7f9bf80cdee0 2023-10-16T08:19:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bf6d9d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 0x7f9bf814f970 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9be800d5f0 tx=0x7f9be800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80125f0 con 0x7f9bf80cdee0 2023-10-16T08:19:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8012750 con 0x7f9bf80cdee0 2023-10-16T08:19:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8012ae0 con 0x7f9bf80cdee0 2023-10-16T08:19:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf815ac50 con 0x7f9bf80cdee0 2023-10-16T08:19:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf815b450 con 0x7f9bf80cdee0 2023-10-16T08:19:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8074010 con 0x7f9bf80cdee0 2023-10-16T08:19:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9be8012c40 con 0x7f9bf80cdee0 2023-10-16T08:19:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bef7fe640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9bd80664d0 0x7f9bd8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.444+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9be8083fe0 con 0x7f9bf80cdee0 2023-10-16T08:19:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.448+0000 7f9beffff640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9bd80664d0 0x7f9bd8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.448+0000 7f9beffff640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9bd80664d0 0x7f9bd8068990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80b5a00 tx=0x7f9be0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:27.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.448+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9be80516c0 con 0x7f9bf80cdee0 2023-10-16T08:19:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.600+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bf815a800 con 0x7f9bd80664d0 2023-10-16T08:19:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.604+0000 7f9bef7fe640 1 -- 172.21.15.32:0/4060475535 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50785 (secure 0 0 0) 0x7f9bf815a800 con 0x7f9bd80664d0 2023-10-16T08:19:27.610 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:19:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9bd80664d0 msgr2=0x7f9bd8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9bd80664d0 0x7f9bd8068990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80b5a00 tx=0x7f9be0009040 comp rx=0 tx=0).stop 2023-10-16T08:19:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 msgr2=0x7f9bf814f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 0x7f9bf814f970 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9be800d5f0 tx=0x7f9be800dab0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 shutdown_connections 2023-10-16T08:19:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f9bd80664d0 0x7f9bd8068990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f9bf815cf00 0x7f9bf81f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bf80d1410 0x7f9bf81f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 --2- 172.21.15.32:0/4060475535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bf80cdee0 0x7f9bf814f970 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 >> 172.21.15.32:0/4060475535 conn(0x7f9bf80b4470 msgr2=0x7f9bf8158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 shutdown_connections 2023-10-16T08:19:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.608+0000 7f9bfd19d640 1 -- 172.21.15.32:0/4060475535 wait complete. 2023-10-16T08:19:27.628 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-10-16T08:19:26.059115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239200,"kb_used_data":6400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735520,"statfs":{"total":773094113280,"available":772849172480,"internally_reserved":0,"allocated":6553600,"data_stored":3577882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001656"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459302+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:18:52.459302+0000","last_peered":"2023-10-16T08:18:52.459302+0000","last_clean":"2023-10-16T08:18:52.459302+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:18:52.459302+0000","last_undegraded":"2023-10-16T08:18:52.459302+0000","last_fullsized":"2023-10-16T08:18:52.459302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1261568,"data_stored":885898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.76500000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91100000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0209999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85399999999999998,"5min":1.03,"15min":1.03},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85399999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":1.056,"15min":1.056},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83899999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80200000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88,"5min":1.127,"15min":1.127},"last":0.753},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.59599999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.626}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68000000000000005,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":1.077,"15min":1.077},"last":0.66800000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.90600000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.91500000000000004,"5min":1.343,"15min":1.343},"last":0.69299999999999995}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97899999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.439},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.008,"15min":1.008},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88100000000000001,"5min":1.036,"15min":1.036},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":296022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89100000000000001,"5min":2.774,"15min":2.774},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.58499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94099999999999995,"5min":1.099,"15min":1.099},"last":0.75}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.504,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92300000000000004,"5min":2.706,"15min":2.706},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.87},"min":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.044,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.89200000000000002},"min":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.79300000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.024,"15min":1.024},"last":0.627},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56299999999999994}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77800000000000002},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.002,"15min":1.002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94799999999999995,"5min":1.075,"15min":1.075},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":311665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85099999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.57299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0089999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.87}]},{"osd":5,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.997,"15min":0.997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81499999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":295426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44400000000000001,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77300000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.042,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.355,"15min":1.355},"last":0.76600000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.60599999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.77700000000000002},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.88700000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93500000000000005,"5min":1.391,"15min":1.391},"last":0.752},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.60799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0169999999999999,"5min":1.3,"15min":1.3},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":295426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.64100000000000001,"15min":0.74299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76500000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.40100000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":4.399,"15min":4.399},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.68000000000000005,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72399999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.71799999999999997,"15min":0.81599999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66500000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.80100000000000005,"15min":0.90300000000000002},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.626}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6599999999999999,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.1799999999999999,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.98799999999999999,"15min":1.0700000000000001},"min":{"1min":0.63100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1519999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.96899999999999997,"15min":1.0489999999999999},"min":{"1min":0.53400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.0489999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":1.0529999999999999},"min":{"1min":0.52400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.1640000000000001,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.97499999999999998,"15min":1.0629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.0649999999999999},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.879,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98999999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.65000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":663552,"data_stored":295460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.434,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438},{"interface":"front","average":{"1min":0.443,"5min":0.46899999999999997,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":1.04,"15min":1.04},"last":0.374}]},{"osd":2,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.622,"5min":1.073,"15min":1.073},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":1.556,"15min":1.556},"last":0.627},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83999999999999997,"15min":0.88400000000000001},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.073,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.93000000000000005,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.091,"5min":1.913,"15min":1.913},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.86599999999999999,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92000000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":902615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76600000000000001,"5min":3.734,"15min":3.734},"last":0.504},{"interface":"front","average":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.44400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78900000000000003,"5min":3.492,"15min":3.492},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68000000000000005,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93500000000000005,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96699999999999997,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:19:27.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:19:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9f50f640 1 Processor -- start 2023-10-16T08:19:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9f50f640 1 -- start start 2023-10-16T08:19:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9f50f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d980c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9f50f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980c7110 con 0x7f0d9815a800 2023-10-16T08:19:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9f50f640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980d04c0 con 0x7f0d980b21c0 2023-10-16T08:19:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9f50f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980cf820 con 0x7f0d980b2630 2023-10-16T08:19:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9d2ab640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d9815a800 0x7f0d980cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57048/0 (socket says 172.21.15.32:57048) 2023-10-16T08:19:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9d2ab640 1 -- 172.21.15.32:0/2213021315 learned_addr learned my addr 172.21.15.32:0/2213021315 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9daac640 1 --2- 172.21.15.32:0/2213021315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d980c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358274088 0 0) 0x7f0d980c7110 con 0x7f0d9815a800 2023-10-16T08:19:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d84003530 con 0x7f0d9815a800 2023-10-16T08:19:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911935476 0 0) 0x7f0d980d04c0 con 0x7f0d980b21c0 2023-10-16T08:19:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d980c7110 con 0x7f0d980b21c0 2023-10-16T08:19:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9daac640 1 -- 172.21.15.32:0/2213021315 >> v1:172.21.15.186:6789/0 conn(0x7f0d980b21c0 legacy=0x7f0d980c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1143647708 0 0) 0x7f0d84003530 con 0x7f0d9815a800 2023-10-16T08:19:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9daac640 1 -- 172.21.15.32:0/2213021315 >> v1:172.21.15.32:6789/0 conn(0x7f0d9815a800 legacy=0x7f0d980cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.772+0000 7f0d9daac640 1 -- 172.21.15.32:0/2213021315 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980b2ac0 con 0x7f0d980b2630 2023-10-16T08:19:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9daac640 1 --2- 172.21.15.32:0/2213021315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d980c49a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009dc0 tx=0x7f0d9001c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1223b5c10454a10c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94001680 con 0x7f0d980b2630 2023-10-16T08:19:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d900a56a0 con 0x7f0d980b2630 2023-10-16T08:19:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d8ffff640 1 -- 172.21.15.32:0/2213021315 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900a5a30 con 0x7f0d980b2630 2023-10-16T08:19:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 -- 172.21.15.32:0/2213021315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 msgr2=0x7f0d980c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/2213021315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d980c49a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009dc0 tx=0x7f0d9001c860 comp rx=0 tx=0).stop 2023-10-16T08:19:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 -- 172.21.15.32:0/2213021315 shutdown_connections 2023-10-16T08:19:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/2213021315 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d980c49a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 -- 172.21.15.32:0/2213021315 >> 172.21.15.32:0/2213021315 conn(0x7f0d980cd1c0 msgr2=0x7f0d980cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 -- 172.21.15.32:0/2213021315 shutdown_connections 2023-10-16T08:19:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.776+0000 7f0d9f50f640 1 -- 172.21.15.32:0/2213021315 wait complete. 2023-10-16T08:19:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 Processor -- start 2023-10-16T08:19:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 -- start start 2023-10-16T08:19:27.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 0x7f0d9815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d98154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d9815a800 0x7f0d981552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d981558f0 con 0x7f0d980b21c0 2023-10-16T08:19:27.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98155bd0 con 0x7f0d9815a800 2023-10-16T08:19:27.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98155eb0 con 0x7f0d980b2630 2023-10-16T08:19:27.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 0x7f0d9815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 0x7f0d9815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35124/0 (socket says 172.21.15.32:35124) 2023-10-16T08:19:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 -- 172.21.15.32:0/3030037773 learned_addr learned my addr 172.21.15.32:0/3030037773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9daac640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d9815a800 0x7f0d981552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9caaa640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d98154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 -- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 msgr2=0x7f0d98154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d98154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 -- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d9815a800 msgr2=0x7f0d981552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d9815a800 0x7f0d981552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 -- 172.21.15.32:0/3030037773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90009a70 con 0x7f0d980b21c0 2023-10-16T08:19:27.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9caaa640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d98154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:27.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9d2ab640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 0x7f0d9815cec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c750 tx=0x7f0d9400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:27.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94005ae0 con 0x7f0d980b21c0 2023-10-16T08:19:27.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d94005c40 con 0x7f0d980b21c0 2023-10-16T08:19:27.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d98156190 con 0x7f0d980b21c0 2023-10-16T08:19:27.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94007070 con 0x7f0d980b21c0 2023-10-16T08:19:27.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.780+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9815d400 con 0x7f0d980b21c0 2023-10-16T08:19:27.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.784+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d98074010 con 0x7f0d980b21c0 2023-10-16T08:19:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.784+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0d940071d0 con 0x7f0d980b21c0 2023-10-16T08:19:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.784+0000 7f0d8dffb640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d7c0664a0 0x7f0d7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.784+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0d94083980 con 0x7f0d980b21c0 2023-10-16T08:19:27.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.784+0000 7f0d9caaa640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d7c0664a0 0x7f0d7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:27.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.784+0000 7f0d9caaa640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d7c0664a0 0x7f0d7c068960 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0d98156a20 tx=0x7f0d88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:27.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.788+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0d94050f20 con 0x7f0d980b21c0 2023-10-16T08:19:27.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.944+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d98155d40 con 0x7f0d7c0664a0 2023-10-16T08:19:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.944+0000 7f0d8dffb640 1 -- 172.21.15.32:0/3030037773 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50785 (secure 0 0 0) 0x7f0d98155d40 con 0x7f0d7c0664a0 2023-10-16T08:19:27.949 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:19:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d7c0664a0 msgr2=0x7f0d7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d7c0664a0 0x7f0d7c068960 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0d98156a20 tx=0x7f0d88009040 comp rx=0 tx=0).stop 2023-10-16T08:19:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 msgr2=0x7f0d9815cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 0x7f0d9815cec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c750 tx=0x7f0d9400cc10 comp rx=0 tx=0).stop 2023-10-16T08:19:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 shutdown_connections 2023-10-16T08:19:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d9815a800 0x7f0d981552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d7c0664a0 0x7f0d7c068960 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d980b2630 0x7f0d98154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 --2- 172.21.15.32:0/3030037773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d980b21c0 0x7f0d9815cec0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 >> 172.21.15.32:0/3030037773 conn(0x7f0d980cd1c0 msgr2=0x7f0d980cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 shutdown_connections 2023-10-16T08:19:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:27.948+0000 7f0d9f50f640 1 -- 172.21.15.32:0/3030037773 wait complete. 2023-10-16T08:19:27.969 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-10-16T08:19:26.059115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239200,"kb_used_data":6400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735520,"statfs":{"total":773094113280,"available":772849172480,"internally_reserved":0,"allocated":6553600,"data_stored":3577882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001656"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459302+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:18:52.459302+0000","last_peered":"2023-10-16T08:18:52.459302+0000","last_clean":"2023-10-16T08:18:52.459302+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:18:52.459302+0000","last_undegraded":"2023-10-16T08:18:52.459302+0000","last_fullsized":"2023-10-16T08:18:52.459302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1261568,"data_stored":885898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.76500000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91100000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0209999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85399999999999998,"5min":1.03,"15min":1.03},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85399999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":1.056,"15min":1.056},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83899999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80200000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88,"5min":1.127,"15min":1.127},"last":0.753},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.59599999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.626}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68000000000000005,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":1.077,"15min":1.077},"last":0.66800000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.90600000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.91500000000000004,"5min":1.343,"15min":1.343},"last":0.69299999999999995}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97899999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.439},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.008,"15min":1.008},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88100000000000001,"5min":1.036,"15min":1.036},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":296022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89100000000000001,"5min":2.774,"15min":2.774},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.58499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94099999999999995,"5min":1.099,"15min":1.099},"last":0.75}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.504,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92300000000000004,"5min":2.706,"15min":2.706},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.87},"min":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.044,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.89200000000000002},"min":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.79300000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.024,"15min":1.024},"last":0.627},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56299999999999994}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77800000000000002},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.002,"15min":1.002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94799999999999995,"5min":1.075,"15min":1.075},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":311665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85099999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.57299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0089999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.87}]},{"osd":5,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.997,"15min":0.997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81499999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":295426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44400000000000001,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77300000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.042,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.355,"15min":1.355},"last":0.76600000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.60599999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.77700000000000002},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.88700000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93500000000000005,"5min":1.391,"15min":1.391},"last":0.752},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.60799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0169999999999999,"5min":1.3,"15min":1.3},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":295426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.64100000000000001,"15min":0.74299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76500000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.40100000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":4.399,"15min":4.399},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.68000000000000005,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72399999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.71799999999999997,"15min":0.81599999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66500000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.80100000000000005,"15min":0.90300000000000002},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.626}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6599999999999999,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.1799999999999999,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.98799999999999999,"15min":1.0700000000000001},"min":{"1min":0.63100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1519999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.96899999999999997,"15min":1.0489999999999999},"min":{"1min":0.53400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.0489999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":1.0529999999999999},"min":{"1min":0.52400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.1640000000000001,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.97499999999999998,"15min":1.0629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.0649999999999999},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.879,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98999999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.65000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":663552,"data_stored":295460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.434,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438},{"interface":"front","average":{"1min":0.443,"5min":0.46899999999999997,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":1.04,"15min":1.04},"last":0.374}]},{"osd":2,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.622,"5min":1.073,"15min":1.073},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":1.556,"15min":1.556},"last":0.627},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83999999999999997,"15min":0.88400000000000001},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.073,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.93000000000000005,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.091,"5min":1.913,"15min":1.913},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.86599999999999999,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92000000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":902615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76600000000000001,"5min":3.734,"15min":3.734},"last":0.504},{"interface":"front","average":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.44400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78900000000000003,"5min":3.492,"15min":3.492},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68000000000000005,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93500000000000005,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96699999999999997,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:19:27.971 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T08:19:27.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:19:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 Processor -- start 2023-10-16T08:19:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 -- start start 2023-10-16T08:19:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd3c06b200 0x7fcd3c06b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06bb20 0x7fcd3c0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c0b1230 con 0x7fcd3c0b1c00 2023-10-16T08:19:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c06c020 con 0x7fcd3c06b200 2023-10-16T08:19:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3b59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b0900 con 0x7fcd3c06bb20 2023-10-16T08:19:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd3c06b200 0x7fcd3c06b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd3c06b200 0x7fcd3c06b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35130/0 (socket says 172.21.15.32:35130) 2023-10-16T08:19:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.112+0000 7fcd3a59c640 1 -- 172.21.15.32:0/1911208512 learned_addr learned my addr 172.21.15.32:0/1911208512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd33fff640 1 --2- 172.21.15.32:0/1911208512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06bb20 0x7fcd3c0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3a59c640 1 -- 172.21.15.32:0/1911208512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06bb20 msgr2=0x7fcd3c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3a59c640 1 --2- 172.21.15.32:0/1911208512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06bb20 0x7fcd3c0b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3a59c640 1 -- 172.21.15.32:0/1911208512 >> v1:172.21.15.186:6789/0 conn(0x7fcd3c0b1c00 legacy=0x7fcd3c0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3a59c640 1 -- 172.21.15.32:0/1911208512 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c0b0be0 con 0x7fcd3c06b200 2023-10-16T08:19:28.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3a59c640 1 --2- 172.21.15.32:0/1911208512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06b200 0x7fcd3c06b5e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c006fd0 tx=0x7fcd2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a21b8402f0a5bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:28.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd39d9b640 1 -- 172.21.15.32:0/1911208512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0a9400 con 0x7fcd3c06b200 2023-10-16T08:19:28.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd39d9b640 1 -- 172.21.15.32:0/1911208512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c0adc50 con 0x7fcd3c06b200 2023-10-16T08:19:28.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd39d9b640 1 -- 172.21.15.32:0/1911208512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0a9780 con 0x7fcd3c06b200 2023-10-16T08:19:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3b59e640 1 -- 172.21.15.32:0/1911208512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06b200 msgr2=0x7fcd3c06b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.116+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/1911208512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06b200 0x7fcd3c06b5e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c006fd0 tx=0x7fcd2c01c520 comp rx=0 tx=0).stop 2023-10-16T08:19:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- 172.21.15.32:0/1911208512 shutdown_connections 2023-10-16T08:19:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/1911208512 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06bb20 0x7fcd3c0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/1911208512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06b200 0x7fcd3c06b5e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- 172.21.15.32:0/1911208512 >> 172.21.15.32:0/1911208512 conn(0x7fcd3c0d0620 msgr2=0x7fcd3c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- 172.21.15.32:0/1911208512 shutdown_connections 2023-10-16T08:19:28.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- 172.21.15.32:0/1911208512 wait complete. 2023-10-16T08:19:28.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 Processor -- start 2023-10-16T08:19:28.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- start start 2023-10-16T08:19:28.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06b200 0x7fcd3c0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 0x7fcd3c14db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0b1c00 0x7fcd3c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c14e0c0 con 0x7fcd3c06bb20 2023-10-16T08:19:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0cb280 con 0x7fcd3c0b1c00 2023-10-16T08:19:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3b59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c15dd50 con 0x7fcd3c06b200 2023-10-16T08:19:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 0x7fcd3c14db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 0x7fcd3c14db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35134/0 (socket says 172.21.15.32:35134) 2023-10-16T08:19:28.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 -- 172.21.15.32:0/2630311131 learned_addr learned my addr 172.21.15.32:0/2630311131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:28.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3a59c640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06b200 0x7fcd3c0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3ad9d640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0b1c00 0x7fcd3c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 -- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06b200 msgr2=0x7fcd3c0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06b200 0x7fcd3c0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 -- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0b1c00 msgr2=0x7fcd3c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0b1c00 0x7fcd3c15d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 -- 172.21.15.32:0/2630311131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c009480 con 0x7fcd3c06bb20 2023-10-16T08:19:28.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd3a59c640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06b200 0x7fcd3c0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:28.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.120+0000 7fcd33fff640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 0x7fcd3c14db80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400d800 tx=0x7fcd2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:28.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400b6c0 con 0x7fcd3c06bb20 2023-10-16T08:19:28.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2400b860 con 0x7fcd3c06bb20 2023-10-16T08:19:28.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400bc10 con 0x7fcd3c06bb20 2023-10-16T08:19:28.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c15dff0 con 0x7fcd3c06bb20 2023-10-16T08:19:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c0cac30 con 0x7fcd3c06bb20 2023-10-16T08:19:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c0cb3f0 con 0x7fcd3c06bb20 2023-10-16T08:19:28.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd2401e460 con 0x7fcd3c06bb20 2023-10-16T08:19:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd32ffd640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcd1c0664a0 0x7fcd1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fcd24084150 con 0x7fcd3c06bb20 2023-10-16T08:19:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.124+0000 7fcd3a59c640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcd1c0664a0 0x7fcd1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.128+0000 7fcd3a59c640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcd1c0664a0 0x7fcd1c068960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0ae000 tx=0x7fcd2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.128+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd24051830 con 0x7fcd3c06bb20 2023-10-16T08:19:28.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.284+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd3c169780 con 0x7fcd1c0664a0 2023-10-16T08:19:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.288+0000 7fcd32ffd640 1 -- 172.21.15.32:0/2630311131 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50785 (secure 0 0 0) 0x7fcd2c003620 con 0x7fcd1c0664a0 2023-10-16T08:19:28.291 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:19:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.288+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcd1c0664a0 msgr2=0x7fcd1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.288+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcd1c0664a0 0x7fcd1c068960 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0ae000 tx=0x7fcd2c008040 comp rx=0 tx=0).stop 2023-10-16T08:19:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.288+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 msgr2=0x7fcd3c14db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.288+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 0x7fcd3c14db80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400d800 tx=0x7fcd2400dcc0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 shutdown_connections 2023-10-16T08:19:28.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd3c0b1c00 0x7fcd3c15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd3c06bb20 0x7fcd3c14db80 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fcd1c0664a0 0x7fcd1c068960 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 --2- 172.21.15.32:0/2630311131 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fcd3c06b200 0x7fcd3c0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 >> 172.21.15.32:0/2630311131 conn(0x7fcd3c0d0620 msgr2=0x7fcd3c14e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 shutdown_connections 2023-10-16T08:19:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.292+0000 7fcd3b59e640 1 -- 172.21.15.32:0/2630311131 wait complete. 2023-10-16T08:19:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2023-10-16T08:19:28.059436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239200,"kb_used_data":6400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735520,"statfs":{"total":773094113280,"available":772849172480,"internally_reserved":0,"allocated":6553600,"data_stored":3577882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14930,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-16T08:18:52.459302+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:18:52.459302+0000","last_peered":"2023-10-16T08:18:52.459302+0000","last_clean":"2023-10-16T08:18:52.459302+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:18:52.459302+0000","last_undegraded":"2023-10-16T08:18:52.459302+0000","last_fullsized":"2023-10-16T08:18:52.459302+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1261568,"data_stored":885898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.76500000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91100000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0209999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85399999999999998,"5min":1.03,"15min":1.03},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85399999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":1.056,"15min":1.056},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83899999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80200000000000005}]},{"osd":6,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88,"5min":1.127,"15min":1.127},"last":0.753},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.59599999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.91500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.626}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68000000000000005,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":1.077,"15min":1.077},"last":0.66800000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.90600000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.91500000000000004,"5min":1.343,"15min":1.343},"last":0.69299999999999995}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97899999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.439},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.008,"15min":1.008},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88100000000000001,"5min":1.036,"15min":1.036},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":296022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89100000000000001,"5min":2.774,"15min":2.774},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.58499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94099999999999995,"5min":1.099,"15min":1.099},"last":0.75}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.504,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92300000000000004,"5min":2.706,"15min":2.706},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.87},"min":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.044,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.89200000000000002},"min":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.79300000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.024,"15min":1.024},"last":0.627},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56299999999999994}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77800000000000002},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.002,"15min":1.002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94799999999999995,"5min":1.075,"15min":1.075},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":311665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.85099999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.57299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0089999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.87}]},{"osd":5,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.997,"15min":0.997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81499999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":295426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44400000000000001,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77300000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.042,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.355,"15min":1.355},"last":0.76600000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.60599999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.77700000000000002},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.88700000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93500000000000005,"5min":1.391,"15min":1.391},"last":0.752},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.60799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0169999999999999,"5min":1.3,"15min":1.3},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":295426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.64100000000000001,"15min":0.74299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76500000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.40100000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":4.399,"15min":4.399},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.68000000000000005,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72399999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.71799999999999997,"15min":0.81599999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66500000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.80100000000000005,"15min":0.90300000000000002},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.626}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6599999999999999,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.1799999999999999,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.98799999999999999,"15min":1.0700000000000001},"min":{"1min":0.63100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1519999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.96899999999999997,"15min":1.0489999999999999},"min":{"1min":0.53400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.0489999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":1.0529999999999999},"min":{"1min":0.52400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.1640000000000001,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.97499999999999998,"15min":1.0629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.0649999999999999},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.879,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98999999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.65000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":663552,"data_stored":295460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.434,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.438},{"interface":"front","average":{"1min":0.443,"5min":0.46899999999999997,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":1.04,"15min":1.04},"last":0.374}]},{"osd":2,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.622,"5min":1.073,"15min":1.073},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":1.556,"15min":1.556},"last":0.627},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83999999999999997,"15min":0.88400000000000001},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.073,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.93000000000000005,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.091,"5min":1.913,"15min":1.913},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.86599999999999999,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92000000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.67400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":902615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76600000000000001,"5min":3.734,"15min":3.734},"last":0.504},{"interface":"front","average":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.44400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78900000000000003,"5min":3.492,"15min":3.492},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68000000000000005,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93500000000000005,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96699999999999997,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.79600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:19:28.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:19:28.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 Processor -- start 2023-10-16T08:19:28.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 -- start start 2023-10-16T08:19:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c06af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd51c15cd60 0x7fd51c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c0d3510 con 0x7fd51c0d0d80 2023-10-16T08:19:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0cd2b0 con 0x7fd51c15cd60 2023-10-16T08:19:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd520808640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0d2f50 con 0x7fd51c15c980 2023-10-16T08:19:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51b59e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd51c0d0d80 0x7fd51c0cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57052/0 (socket says 172.21.15.32:57052) 2023-10-16T08:19:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd51c15cd60 0x7fd51c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51b59e640 1 -- 172.21.15.32:0/95958322 learned_addr learned my addr 172.21.15.32:0/95958322 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51ad9d640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c06af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51a59c640 1 -- 172.21.15.32:0/95958322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 msgr2=0x7fd51c06af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51a59c640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c06af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51a59c640 1 -- 172.21.15.32:0/95958322 >> v1:172.21.15.32:6789/0 conn(0x7fd51c0d0d80 legacy=0x7fd51c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51a59c640 1 -- 172.21.15.32:0/95958322 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0d3230 con 0x7fd51c15cd60 2023-10-16T08:19:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd51ad9d640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c06af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.456+0000 7fd519d9b640 1 -- 172.21.15.32:0/95958322 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850789426 0 0) 0x7fd5140014a0 con 0x7fd51c0d0d80 2023-10-16T08:19:28.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd51a59c640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c0b1b90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd5100097c0 tx=0x7fd51001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f941d2f8bc07255 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd519d9b640 1 -- 172.21.15.32:0/95958322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100ad640 con 0x7fd51c15cd60 2023-10-16T08:19:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd519d9b640 1 -- 172.21.15.32:0/95958322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100ad7a0 con 0x7fd51c15cd60 2023-10-16T08:19:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd519d9b640 1 -- 172.21.15.32:0/95958322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100adb30 con 0x7fd51c15cd60 2023-10-16T08:19:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 -- 172.21.15.32:0/95958322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 msgr2=0x7fd51c0b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c0b1b90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd5100097c0 tx=0x7fd51001c480 comp rx=0 tx=0).stop 2023-10-16T08:19:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 -- 172.21.15.32:0/95958322 shutdown_connections 2023-10-16T08:19:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c0b1b90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 --2- 172.21.15.32:0/95958322 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c06af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 -- 172.21.15.32:0/95958322 >> 172.21.15.32:0/95958322 conn(0x7fd51c09ba80 msgr2=0x7fd51c0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 -- 172.21.15.32:0/95958322 shutdown_connections 2023-10-16T08:19:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.460+0000 7fd520808640 1 -- 172.21.15.32:0/95958322 wait complete. 2023-10-16T08:19:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 Processor -- start 2023-10-16T08:19:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- start start 2023-10-16T08:19:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c0d0d80 0x7fd51c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0c7d40 con 0x7fd51c15cd60 2023-10-16T08:19:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0c8020 con 0x7fd51c0d0d80 2023-10-16T08:19:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0c8300 con 0x7fd51c15c980 2023-10-16T08:19:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35150/0 (socket says 172.21.15.32:35150) 2023-10-16T08:19:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 -- 172.21.15.32:0/3405651685 learned_addr learned my addr 172.21.15.32:0/3405651685 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51a59c640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51ad9d640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c0d0d80 0x7fd51c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 -- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 msgr2=0x7fd51c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c0c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 -- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c0d0d80 msgr2=0x7fd51c0afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c0d0d80 0x7fd51c0afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 -- 172.21.15.32:0/3405651685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510009420 con 0x7fd51c15cd60 2023-10-16T08:19:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd51b59e640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c154830 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c7e0 tx=0x7fd51400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400a640 con 0x7fd51c15cd60 2023-10-16T08:19:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400a7c0 con 0x7fd51c15cd60 2023-10-16T08:19:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c0d1720 con 0x7fd51c15cd60 2023-10-16T08:19:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400abb0 con 0x7fd51c15cd60 2023-10-16T08:19:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c0c88c0 con 0x7fd51c15cd60 2023-10-16T08:19:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.464+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c0c8ab0 con 0x7fd51c15cd60 2023-10-16T08:19:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.468+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd51400ad10 con 0x7fd51c15cd60 2023-10-16T08:19:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.468+0000 7fd5037fe640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd5080664a0 0x7fd508068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.468+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd514083410 con 0x7fd51c15cd60 2023-10-16T08:19:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.468+0000 7fd51ad9d640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd5080664a0 0x7fd508068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.468+0000 7fd51ad9d640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd5080664a0 0x7fd508068960 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd50c006fd0 tx=0x7fd50c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:28.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.472+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd514050a50 con 0x7fd51c15cd60 2023-10-16T08:19:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.628+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd51c04e9c0 con 0x7fd51c15cd60 2023-10-16T08:19:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.628+0000 7fd5037fe640 1 -- 172.21.15.32:0/3405651685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fd514054070 con 0x7fd51c15cd60 2023-10-16T08:19:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:28.632 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"943c836a-5c4c-4cb7-a190-9643b3476f99","created":"2023-10-16T08:15:56.937002+0000","modified":"2023-10-16T08:19:25.195328+0000","last_up_change":"2023-10-16T08:16:08.525788+0000","last_in_change":"2023-10-16T08:15:59.468950+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:16:10.073833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a1b4a42-2706-464f-a2b4-b23c887af4ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6809","nonce":1724855588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6811","nonce":1724855588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6815","nonce":1724855588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":1724855588},{"type":"v1","addr":"172.21.15.32:6813","nonce":1724855588}]},"public_addr":"172.21.15.32:6809/1724855588","cluster_addr":"172.21.15.32:6811/1724855588","heartbeat_back_addr":"172.21.15.32:6815/1724855588","heartbeat_front_addr":"172.21.15.32:6813/1724855588","state":["exists","up"]},{"osd":1,"uuid":"f773ecc7-b793-44c4-ae48-592c08df1272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6801","nonce":1484869129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6803","nonce":1484869129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6807","nonce":1484869129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1484869129},{"type":"v1","addr":"172.21.15.32:6805","nonce":1484869129}]},"public_addr":"172.21.15.32:6801/1484869129","cluster_addr":"172.21.15.32:6803/1484869129","heartbeat_back_addr":"172.21.15.32:6807/1484869129","heartbeat_front_addr":"172.21.15.32:6805/1484869129","state":["exists","up"]},{"osd":2,"uuid":"68eab466-4b6b-406f-86f7-6414a3c00a32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6817","nonce":1665192800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6819","nonce":1665192800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6823","nonce":1665192800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1665192800},{"type":"v1","addr":"172.21.15.32:6821","nonce":1665192800}]},"public_addr":"172.21.15.32:6817/1665192800","cluster_addr":"172.21.15.32:6819/1665192800","heartbeat_back_addr":"172.21.15.32:6823/1665192800","heartbeat_front_addr":"172.21.15.32:6821/1665192800","state":["exists","up"]},{"osd":3,"uuid":"7376c799-427b-4894-9d89-d4ffcff9510e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6825","nonce":2556335256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6827","nonce":2556335256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6831","nonce":2556335256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2556335256},{"type":"v1","addr":"172.21.15.32:6829","nonce":2556335256}]},"public_addr":"172.21.15.32:6825/2556335256","cluster_addr":"172.21.15.32:6827/2556335256","heartbeat_back_addr":"172.21.15.32:6831/2556335256","heartbeat_front_addr":"172.21.15.32:6829/2556335256","state":["exists","up"]},{"osd":4,"uuid":"d004f570-0acb-458d-a7e3-d99e637baa3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6825","nonce":2991882832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6827","nonce":2991882832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6831","nonce":2991882832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2991882832},{"type":"v1","addr":"172.21.15.186:6829","nonce":2991882832}]},"public_addr":"172.21.15.186:6825/2991882832","cluster_addr":"172.21.15.186:6827/2991882832","heartbeat_back_addr":"172.21.15.186:6831/2991882832","heartbeat_front_addr":"172.21.15.186:6829/2991882832","state":["exists","up"]},{"osd":5,"uuid":"42ff7c69-5065-4349-98ef-e35cd5a7a0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6801","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6803","nonce":3150472125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6807","nonce":3150472125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6817","nonce":3150472125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3150472125},{"type":"v1","addr":"172.21.15.186:6812","nonce":3150472125}]},"public_addr":"172.21.15.186:6803/3150472125","cluster_addr":"172.21.15.186:6807/3150472125","heartbeat_back_addr":"172.21.15.186:6817/3150472125","heartbeat_front_addr":"172.21.15.186:6812/3150472125","state":["exists","up"]},{"osd":6,"uuid":"2e1f98cc-738b-4c3d-b766-7a38b271ddf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6808","nonce":1135936083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6811","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6815","nonce":1135936083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6823","nonce":1135936083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":1135936083},{"type":"v1","addr":"172.21.15.186:6820","nonce":1135936083}]},"public_addr":"172.21.15.186:6808/1135936083","cluster_addr":"172.21.15.186:6815/1135936083","heartbeat_back_addr":"172.21.15.186:6823/1135936083","heartbeat_front_addr":"172.21.15.186:6820/1135936083","state":["exists","up"]},{"osd":7,"uuid":"6ef75fcd-dce3-4d95-b560-d54fc2be1226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6802","nonce":403837937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6805","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6809","nonce":403837937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6819","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6821","nonce":403837937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6813","nonce":403837937},{"type":"v1","addr":"172.21.15.186:6816","nonce":403837937}]},"public_addr":"172.21.15.186:6802/403837937","cluster_addr":"172.21.15.186:6809/403837937","heartbeat_back_addr":"172.21.15.186:6821/403837937","heartbeat_front_addr":"172.21.15.186:6816/403837937","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.926940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.923348+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:04.892556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.228981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.214056+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.044905+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.192436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:16:05.210874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6834/3873719355":"2023-10-17T08:19:22.187790+0000","172.21.15.32:6833/3873719355":"2023-10-17T08:19:22.187790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:19:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd5080664a0 msgr2=0x7fd508068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd5080664a0 0x7fd508068960 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd50c006fd0 tx=0x7fd50c008040 comp rx=0 tx=0).stop 2023-10-16T08:19:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 msgr2=0x7fd51c154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c154830 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c7e0 tx=0x7fd51400cca0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 shutdown_connections 2023-10-16T08:19:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c15cd60 0x7fd51c154830 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd5080664a0 0x7fd508068960 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd51c15c980 0x7fd51c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 --2- 172.21.15.32:0/3405651685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd51c0d0d80 0x7fd51c0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 >> 172.21.15.32:0/3405651685 conn(0x7fd51c09ba80 msgr2=0x7fd51c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 shutdown_connections 2023-10-16T08:19:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:28.632+0000 7fd520808640 1 -- 172.21.15.32:0/3405651685 wait complete. 2023-10-16T08:19:29.653 INFO:tasks.ceph:Scrubbing osd.0 2023-10-16T08:19:29.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:29.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.804+0000 7fa8b41b8640 1 Processor -- start 2023-10-16T08:19:29.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b41b8640 1 -- start start 2023-10-16T08:19:29.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b41b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8ac1595a0 0x7fa8ac06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b41b8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac159980 0x7fa8ac06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b41b8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0c8a20 con 0x7fa8ac0b3bc0 2023-10-16T08:19:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b41b8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac15e300 con 0x7fa8ac1595a0 2023-10-16T08:19:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b41b8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0c8c10 con 0x7fa8ac159980 2023-10-16T08:19:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8ac1595a0 0x7fa8ac06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8ac1595a0 0x7fa8ac06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35160/0 (socket says 172.21.15.32:35160) 2023-10-16T08:19:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 -- 172.21.15.32:0/495860434 learned_addr learned my addr 172.21.15.32:0/495860434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1753640 1 --2- 172.21.15.32:0/495860434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac159980 0x7fa8ac06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 -- 172.21.15.32:0/495860434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac159980 msgr2=0x7fa8ac06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 --2- 172.21.15.32:0/495860434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac159980 0x7fa8ac06c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 -- 172.21.15.32:0/495860434 >> v1:172.21.15.186:6789/0 conn(0x7fa8ac0b3bc0 legacy=0x7fa8ac0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.808+0000 7fa8b1f54640 1 -- 172.21.15.32:0/495860434 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0c8ef0 con 0x7fa8ac1595a0 2023-10-16T08:19:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b1f54640 1 --2- 172.21.15.32:0/495860434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac1595a0 0x7fa8ac06c4b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800bb10 tx=0x7fa8a801eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7fa225eae0c022 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b0f52640 1 -- 172.21.15.32:0/495860434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80a7c20 con 0x7fa8ac1595a0 2023-10-16T08:19:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b0f52640 1 -- 172.21.15.32:0/495860434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a80a7d80 con 0x7fa8ac1595a0 2023-10-16T08:19:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b0f52640 1 -- 172.21.15.32:0/495860434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80af660 con 0x7fa8ac1595a0 2023-10-16T08:19:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 -- 172.21.15.32:0/495860434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac1595a0 msgr2=0x7fa8ac06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 --2- 172.21.15.32:0/495860434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac1595a0 0x7fa8ac06c4b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800bb10 tx=0x7fa8a801eb50 comp rx=0 tx=0).stop 2023-10-16T08:19:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 -- 172.21.15.32:0/495860434 shutdown_connections 2023-10-16T08:19:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 --2- 172.21.15.32:0/495860434 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac159980 0x7fa8ac06c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 --2- 172.21.15.32:0/495860434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac1595a0 0x7fa8ac06c4b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 -- 172.21.15.32:0/495860434 >> 172.21.15.32:0/495860434 conn(0x7fa8ac06a820 msgr2=0x7fa8ac0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 -- 172.21.15.32:0/495860434 shutdown_connections 2023-10-16T08:19:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 -- 172.21.15.32:0/495860434 wait complete. 2023-10-16T08:19:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 Processor -- start 2023-10-16T08:19:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.812+0000 7fa8b41b8640 1 -- start start 2023-10-16T08:19:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac0b3bc0 0x7fa8ac147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8ac1595a0 0x7fa8ac147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac148270 0x7fa8ac14c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac154790 con 0x7fa8ac148270 2023-10-16T08:19:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac153b80 con 0x7fa8ac1595a0 2023-10-16T08:19:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac153e60 con 0x7fa8ac0b3bc0 2023-10-16T08:19:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac148270 0x7fa8ac14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac148270 0x7fa8ac14c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35164/0 (socket says 172.21.15.32:35164) 2023-10-16T08:19:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 -- 172.21.15.32:0/2616305965 learned_addr learned my addr 172.21.15.32:0/2616305965 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b1f54640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac0b3bc0 0x7fa8ac147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b1753640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8ac1595a0 0x7fa8ac147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 -- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac0b3bc0 msgr2=0x7fa8ac147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fa8ac0b3bc0 0x7fa8ac147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 -- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8ac1595a0 msgr2=0x7fa8ac147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8ac1595a0 0x7fa8ac147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 -- 172.21.15.32:0/2616305965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a800b770 con 0x7fa8ac148270 2023-10-16T08:19:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b2755640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac148270 0x7fa8ac14c390 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c770 tx=0x7fa8a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4003b80 con 0x7fa8ac148270 2023-10-16T08:19:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a4003ce0 con 0x7fa8ac148270 2023-10-16T08:19:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400aa90 con 0x7fa8ac148270 2023-10-16T08:19:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac154140 con 0x7fa8ac148270 2023-10-16T08:19:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac153870 con 0x7fa8ac148270 2023-10-16T08:19:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8ac0c9790 con 0x7fa8ac148270 2023-10-16T08:19:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.816+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa8a400abf0 con 0x7fa8ac148270 2023-10-16T08:19:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8a27fc640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa8900664a0 0x7fa890068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa8a40839c0 con 0x7fa8ac148270 2023-10-16T08:19:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8a27fc640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7fa89006c7a0 0x7fa89006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 --> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa89006f2b0 con 0x7fa89006c7a0 2023-10-16T08:19:29.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a40882a0 con 0x7fa8ac148270 2023-10-16T08:19:29.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8b1f54640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa8900664a0 0x7fa890068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8b1753640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7fa89006c7a0 0x7fa89006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:29.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8b1f54640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa8900664a0 0x7fa890068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80b1000 tx=0x7fa8a80a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:29.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8b1753640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7fa89006c7a0 0x7fa89006ebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.820+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== osd.0 v2:172.21.15.32:6808/1724855588 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fa89006f2b0 con 0x7fa89006c7a0 2023-10-16T08:19:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.832+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 --> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa8ac15a540 con 0x7fa89006c7a0 2023-10-16T08:19:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.832+0000 7fa8a27fc640 1 -- 172.21.15.32:0/2616305965 <== osd.0 v2:172.21.15.32:6808/1724855588 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa8ac15a540 con 0x7fa89006c7a0 2023-10-16T08:19:29.837 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:29.837 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:29.837 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7fa89006c7a0 msgr2=0x7fa89006ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:6808/1724855588,v1:172.21.15.32:6809/1724855588] conn(0x7fa89006c7a0 0x7fa89006ebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:29.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa8900664a0 msgr2=0x7fa890068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fa8900664a0 0x7fa890068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80b1000 tx=0x7fa8a80a9070 comp rx=0 tx=0).stop 2023-10-16T08:19:29.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac148270 msgr2=0x7fa8ac14c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:29.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 --2- 172.21.15.32:0/2616305965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8ac148270 0x7fa8ac14c390 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c770 tx=0x7fa8a400cc30 comp rx=0 tx=0).stop 2023-10-16T08:19:29.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b2755640 1 -- 172.21.15.32:0/2616305965 reap_dead start 2023-10-16T08:19:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 shutdown_connections 2023-10-16T08:19:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 >> 172.21.15.32:0/2616305965 conn(0x7fa8ac06a820 msgr2=0x7fa8ac0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 shutdown_connections 2023-10-16T08:19:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.836+0000 7fa8b41b8640 1 -- 172.21.15.32:0/2616305965 wait complete. 2023-10-16T08:19:29.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-16T08:19:30.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.996+0000 7f4cc7679640 1 Processor -- start 2023-10-16T08:19:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:29.996+0000 7f4cc7679640 1 -- start start 2023-10-16T08:19:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc7679640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f4cc0164520 0x7f4cc00ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc7679640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cc00b07f0 0x7f4cc00cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc7679640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc00c96f0 con 0x7f4cc00ad960 2023-10-16T08:19:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc7679640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc0164900 con 0x7f4cc00b07f0 2023-10-16T08:19:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc7679640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc00cb950 con 0x7f4cc0164520 2023-10-16T08:19:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cc00b07f0 0x7f4cc00cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cc00b07f0 0x7f4cc00cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35166/0 (socket says 172.21.15.32:35166) 2023-10-16T08:19:30.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 -- 172.21.15.32:0/3267842346 learned_addr learned my addr 172.21.15.32:0/3267842346 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 -- 172.21.15.32:0/3267842346 >> v2:172.21.15.186:3300/0 conn(0x7f4cc0164520 msgr2=0x7f4cc00ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:30.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5415640 1 --2- 172.21.15.32:0/3267842346 >> v2:172.21.15.186:3300/0 conn(0x7f4cc0164520 0x7f4cc00ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 --2- 172.21.15.32:0/3267842346 >> v2:172.21.15.186:3300/0 conn(0x7f4cc0164520 0x7f4cc00ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 -- 172.21.15.32:0/3267842346 >> v1:172.21.15.186:6789/0 conn(0x7f4cc00ad960 legacy=0x7f4cc00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5c16640 1 -- 172.21.15.32:0/3267842346 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00cbed0 con 0x7f4cc00b07f0 2023-10-16T08:19:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.000+0000 7f4cc5415640 1 --2- 172.21.15.32:0/3267842346 >> v2:172.21.15.186:3300/0 conn(0x7f4cc0164520 0x7f4cc00ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:30.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc5c16640 1 --2- 172.21.15.32:0/3267842346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc00b07f0 0x7f4cc00cc870 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00cf6f0 tx=0x7f4cb801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0f70248cdd2f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cb7fff640 1 -- 172.21.15.32:0/3267842346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80a5840 con 0x7f4cc00b07f0 2023-10-16T08:19:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cb7fff640 1 -- 172.21.15.32:0/3267842346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb80a59a0 con 0x7f4cc00b07f0 2023-10-16T08:19:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cb7fff640 1 -- 172.21.15.32:0/3267842346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80a5d30 con 0x7f4cc00b07f0 2023-10-16T08:19:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 -- 172.21.15.32:0/3267842346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc00b07f0 msgr2=0x7f4cc00cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 --2- 172.21.15.32:0/3267842346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc00b07f0 0x7f4cc00cc870 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00cf6f0 tx=0x7f4cb801c960 comp rx=0 tx=0).stop 2023-10-16T08:19:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 -- 172.21.15.32:0/3267842346 shutdown_connections 2023-10-16T08:19:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 --2- 172.21.15.32:0/3267842346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc00b07f0 0x7f4cc00cc870 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 --2- 172.21.15.32:0/3267842346 >> v2:172.21.15.186:3300/0 conn(0x7f4cc0164520 0x7f4cc00ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 -- 172.21.15.32:0/3267842346 >> 172.21.15.32:0/3267842346 conn(0x7f4cc006c190 msgr2=0x7f4cc006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 -- 172.21.15.32:0/3267842346 shutdown_connections 2023-10-16T08:19:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 -- 172.21.15.32:0/3267842346 wait complete. 2023-10-16T08:19:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 Processor -- start 2023-10-16T08:19:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.004+0000 7f4cc7679640 1 -- start start 2023-10-16T08:19:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc00ad960 0x7f4cc0153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc00b07f0 0x7f4cc015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 0x7f4cc00af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc0153650 con 0x7f4cc0164520 2023-10-16T08:19:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc015e4b0 con 0x7f4cc00ad960 2023-10-16T08:19:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00c4ef0 con 0x7f4cc00b07f0 2023-10-16T08:19:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 0x7f4cc00af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 0x7f4cc00af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35172/0 (socket says 172.21.15.32:35172) 2023-10-16T08:19:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 -- 172.21.15.32:0/2476969368 learned_addr learned my addr 172.21.15.32:0/2476969368 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5415640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc00ad960 0x7f4cc0153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc4c14640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc00b07f0 0x7f4cc015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 -- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc00b07f0 msgr2=0x7f4cc015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc00b07f0 0x7f4cc015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 -- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc00ad960 msgr2=0x7f4cc0153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc00ad960 0x7f4cc0153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 -- 172.21.15.32:0/2476969368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8009b30 con 0x7f4cc0164520 2023-10-16T08:19:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc5c16640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 0x7f4cc00af930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4cb801a590 tx=0x7f4cb8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80b8670 con 0x7f4cc0164520 2023-10-16T08:19:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb80b87d0 con 0x7f4cc0164520 2023-10-16T08:19:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80b8b80 con 0x7f4cc0164520 2023-10-16T08:19:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc00c20c0 con 0x7f4cc0164520 2023-10-16T08:19:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc0154ff0 con 0x7f4cc0164520 2023-10-16T08:19:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc0074010 con 0x7f4cc0164520 2023-10-16T08:19:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.008+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4cb80b8ce0 con 0x7f4cc0164520 2023-10-16T08:19:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.012+0000 7f4cb5ffb640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ca40664a0 0x7f4ca4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.012+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4cb811f2b0 con 0x7f4cc0164520 2023-10-16T08:19:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.012+0000 7f4cc5415640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ca40664a0 0x7f4ca4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.012+0000 7f4cc5415640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ca40664a0 0x7f4ca4068960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006fd0 tx=0x7f4cbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.016+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4cb80eca10 con 0x7f4cc0164520 2023-10-16T08:19:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.172+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc016de20 con 0x7f4ca40664a0 2023-10-16T08:19:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.172+0000 7f4cb5ffb640 1 -- 172.21.15.32:0/2476969368 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4cbc00c050 con 0x7f4ca40664a0 2023-10-16T08:19:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2023-10-16T08:19:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ca40664a0 msgr2=0x7f4ca4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ca40664a0 0x7f4ca4068960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006fd0 tx=0x7f4cbc008040 comp rx=0 tx=0).stop 2023-10-16T08:19:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 msgr2=0x7f4cc00af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 0x7f4cc00af930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4cb801a590 tx=0x7f4cb8019700 comp rx=0 tx=0).stop 2023-10-16T08:19:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 shutdown_connections 2023-10-16T08:19:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cc0164520 0x7f4cc00af930 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f4ca40664a0 0x7f4ca4068960 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f4cc00b07f0 0x7f4cc015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 --2- 172.21.15.32:0/2476969368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc00ad960 0x7f4cc0153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 >> 172.21.15.32:0/2476969368 conn(0x7f4cc006c190 msgr2=0x7f4cc0161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 shutdown_connections 2023-10-16T08:19:30.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.176+0000 7f4cc7679640 1 -- 172.21.15.32:0/2476969368 wait complete. 2023-10-16T08:19:30.196 INFO:tasks.ceph:Scrubbing osd.1 2023-10-16T08:19:30.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.340+0000 7f33ad654640 1 Processor -- start 2023-10-16T08:19:30.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33ad654640 1 -- start start 2023-10-16T08:19:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33ad654640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33a81595a0 0x7f33a806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33ad654640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a8159980 0x7f33a806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33ad654640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a80c8a20 con 0x7f33a80b3bc0 2023-10-16T08:19:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33ad654640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a815e300 con 0x7f33a81595a0 2023-10-16T08:19:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33ad654640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80c8c10 con 0x7f33a8159980 2023-10-16T08:19:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33a81595a0 0x7f33a806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33a81595a0 0x7f33a806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35176/0 (socket says 172.21.15.32:35176) 2023-10-16T08:19:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 -- 172.21.15.32:0/3526941529 learned_addr learned my addr 172.21.15.32:0/3526941529 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f339ffff640 1 --2- 172.21.15.32:0/3526941529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a8159980 0x7f33a806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 -- 172.21.15.32:0/3526941529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a8159980 msgr2=0x7f33a806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 --2- 172.21.15.32:0/3526941529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a8159980 0x7f33a806c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 -- 172.21.15.32:0/3526941529 >> v1:172.21.15.186:6789/0 conn(0x7f33a80b3bc0 legacy=0x7f33a80cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.344+0000 7f33a6ffd640 1 -- 172.21.15.32:0/3526941529 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80c8ef0 con 0x7f33a81595a0 2023-10-16T08:19:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33a6ffd640 1 --2- 172.21.15.32:0/3526941529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a81595a0 0x7f33a806c4b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3398009870 tx=0x7f339801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57547af9795ce7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33a67fc640 1 -- 172.21.15.32:0/3526941529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980ab680 con 0x7f33a81595a0 2023-10-16T08:19:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33a67fc640 1 -- 172.21.15.32:0/3526941529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980ab7e0 con 0x7f33a81595a0 2023-10-16T08:19:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33a67fc640 1 -- 172.21.15.32:0/3526941529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980abb70 con 0x7f33a81595a0 2023-10-16T08:19:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 -- 172.21.15.32:0/3526941529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a81595a0 msgr2=0x7f33a806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 --2- 172.21.15.32:0/3526941529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a81595a0 0x7f33a806c4b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3398009870 tx=0x7f339801c480 comp rx=0 tx=0).stop 2023-10-16T08:19:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 -- 172.21.15.32:0/3526941529 shutdown_connections 2023-10-16T08:19:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 --2- 172.21.15.32:0/3526941529 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a8159980 0x7f33a806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 --2- 172.21.15.32:0/3526941529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a81595a0 0x7f33a806c4b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 -- 172.21.15.32:0/3526941529 >> 172.21.15.32:0/3526941529 conn(0x7f33a806a820 msgr2=0x7f33a80b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 -- 172.21.15.32:0/3526941529 shutdown_connections 2023-10-16T08:19:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 -- 172.21.15.32:0/3526941529 wait complete. 2023-10-16T08:19:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 Processor -- start 2023-10-16T08:19:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.348+0000 7f33ad654640 1 -- start start 2023-10-16T08:19:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a80b3bc0 0x7f33a8147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a81595a0 0x7f33a81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a8148150 0x7f33a8155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8147ea0 con 0x7f33a8148150 2023-10-16T08:19:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81512e0 con 0x7f33a80b3bc0 2023-10-16T08:19:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8150d20 con 0x7f33a81595a0 2023-10-16T08:19:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a8148150 0x7f33a8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a6ffd640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a80b3bc0 0x7f33a8147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a8148150 0x7f33a8155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35180/0 (socket says 172.21.15.32:35180) 2023-10-16T08:19:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 -- 172.21.15.32:0/153393869 learned_addr learned my addr 172.21.15.32:0/153393869 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f339ffff640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a81595a0 0x7f33a81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 -- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a81595a0 msgr2=0x7f33a81503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f33a81595a0 0x7f33a81503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 -- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a80b3bc0 msgr2=0x7f33a8147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a80b3bc0 0x7f33a8147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 -- 172.21.15.32:0/153393869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33980094d0 con 0x7f33a8148150 2023-10-16T08:19:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a6ffd640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33a80b3bc0 0x7f33a8147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33a77fe640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a8148150 0x7f33a8155490 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c770 tx=0x7f33a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0003b80 con 0x7f33a8148150 2023-10-16T08:19:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a0003ce0 con 0x7f33a8148150 2023-10-16T08:19:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000aa90 con 0x7f33a8148150 2023-10-16T08:19:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a8151000 con 0x7f33a8148150 2023-10-16T08:19:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a815aa60 con 0x7f33a8148150 2023-10-16T08:19:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.352+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f33a80c9790 con 0x7f33a8148150 2023-10-16T08:19:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f33a000abf0 con 0x7f33a8148150 2023-10-16T08:19:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33880664a0 0x7f3388068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f33a00839c0 con 0x7f33a8148150 2023-10-16T08:19:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7f338806c7a0 0x7f338806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 --> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f338806f2b0 con 0x7f338806c7a0 2023-10-16T08:19:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a00882a0 con 0x7f33a8148150 2023-10-16T08:19:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f33a6ffd640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33880664a0 0x7f3388068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339ffff640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7f338806c7a0 0x7f338806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339ffff640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7f338806c7a0 0x7f338806ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f33a6ffd640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33880664a0 0x7f3388068960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f33980af000 tx=0x7f33980b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.356+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== osd.1 v2:172.21.15.32:6800/1484869129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f338806f2b0 con 0x7f338806c7a0 2023-10-16T08:19:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.368+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 --> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f33a804e170 con 0x7f338806c7a0 2023-10-16T08:19:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.368+0000 7f339f7fe640 1 -- 172.21.15.32:0/153393869 <== osd.1 v2:172.21.15.32:6800/1484869129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f33a804e170 con 0x7f338806c7a0 2023-10-16T08:19:30.373 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:30.373 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:30.373 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:30.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7f338806c7a0 msgr2=0x7f338806ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:6800/1484869129,v1:172.21.15.32:6801/1484869129] conn(0x7f338806c7a0 0x7f338806ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33880664a0 msgr2=0x7f3388068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f33880664a0 0x7f3388068960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f33980af000 tx=0x7f33980b6040 comp rx=0 tx=0).stop 2023-10-16T08:19:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a8148150 msgr2=0x7f33a8155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 --2- 172.21.15.32:0/153393869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a8148150 0x7f33a8155490 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c770 tx=0x7f33a000cc30 comp rx=0 tx=0).stop 2023-10-16T08:19:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33a77fe640 1 -- 172.21.15.32:0/153393869 reap_dead start 2023-10-16T08:19:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 shutdown_connections 2023-10-16T08:19:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 >> 172.21.15.32:0/153393869 conn(0x7f33a806a820 msgr2=0x7f33a80b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 shutdown_connections 2023-10-16T08:19:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.372+0000 7f33ad654640 1 -- 172.21.15.32:0/153393869 wait complete. 2023-10-16T08:19:30.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-16T08:19:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.532+0000 7f3946dab640 1 Processor -- start 2023-10-16T08:19:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.532+0000 7f3946dab640 1 -- start start 2023-10-16T08:19:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3946dab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3940159ba0 0x7f39400cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3946dab640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39400c8cb0 0x7f39400cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3946dab640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39400723d0 con 0x7f39401597c0 2023-10-16T08:19:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3946dab640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39400717c0 con 0x7f39400c8cb0 2023-10-16T08:19:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3946dab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3940071aa0 con 0x7f3940159ba0 2023-10-16T08:19:30.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3945348640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39400c8cb0 0x7f39400cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3944b47640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39401597c0 0x7f39400ad6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57066/0 (socket says 172.21.15.32:57066) 2023-10-16T08:19:30.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3944b47640 1 -- 172.21.15.32:0/328513845 learned_addr learned my addr 172.21.15.32:0/328513845 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f393ffff640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3940159ba0 0x7f39400cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3945348640 1 -- 172.21.15.32:0/328513845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3940159ba0 msgr2=0x7f39400cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3945348640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3940159ba0 0x7f39400cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3945348640 1 -- 172.21.15.32:0/328513845 >> v1:172.21.15.32:6789/0 conn(0x7f39401597c0 legacy=0x7f39400ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f3945348640 1 -- 172.21.15.32:0/328513845 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940071d80 con 0x7f39400c8cb0 2023-10-16T08:19:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f393ffff640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3940159ba0 0x7f39400cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.536+0000 7f393f7fe640 1 -- 172.21.15.32:0/328513845 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806536892 0 0) 0x7f39400723d0 con 0x7f39401597c0 2023-10-16T08:19:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3945348640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39400c8cb0 0x7f39400cb350 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f393801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfb9151f217b54e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f393f7fe640 1 -- 172.21.15.32:0/328513845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380a5980 con 0x7f39400c8cb0 2023-10-16T08:19:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f393f7fe640 1 -- 172.21.15.32:0/328513845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39380a5ae0 con 0x7f39400c8cb0 2023-10-16T08:19:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f393f7fe640 1 -- 172.21.15.32:0/328513845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380a5e70 con 0x7f39400c8cb0 2023-10-16T08:19:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 -- 172.21.15.32:0/328513845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39400c8cb0 msgr2=0x7f39400cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39400c8cb0 0x7f39400cb350 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f393801c920 comp rx=0 tx=0).stop 2023-10-16T08:19:30.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 -- 172.21.15.32:0/328513845 shutdown_connections 2023-10-16T08:19:30.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39400c8cb0 0x7f39400cb350 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 --2- 172.21.15.32:0/328513845 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f3940159ba0 0x7f39400cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 -- 172.21.15.32:0/328513845 >> 172.21.15.32:0/328513845 conn(0x7f394006c190 msgr2=0x7f394006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 -- 172.21.15.32:0/328513845 shutdown_connections 2023-10-16T08:19:30.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 -- 172.21.15.32:0/328513845 wait complete. 2023-10-16T08:19:30.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 Processor -- start 2023-10-16T08:19:30.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.540+0000 7f3946dab640 1 -- start start 2023-10-16T08:19:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39400c8cb0 0x7f39400c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39401597c0 0x7f394014a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 0x7f394014ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39400c6240 con 0x7f3940159ba0 2023-10-16T08:19:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39400c7fe0 con 0x7f39400c8cb0 2023-10-16T08:19:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400c5f60 con 0x7f39401597c0 2023-10-16T08:19:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 0x7f394014ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 0x7f394014ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35198/0 (socket says 172.21.15.32:35198) 2023-10-16T08:19:30.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 -- 172.21.15.32:0/2497786070 learned_addr learned my addr 172.21.15.32:0/2497786070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3944b47640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39400c8cb0 0x7f39400c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f393ffff640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39401597c0 0x7f394014a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 -- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39401597c0 msgr2=0x7f394014a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39401597c0 0x7f394014a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 -- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39400c8cb0 msgr2=0x7f39400c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39400c8cb0 0x7f39400c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 -- 172.21.15.32:0/2497786070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009a70 con 0x7f3940159ba0 2023-10-16T08:19:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3945348640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 0x7f394014ba90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f39380196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380ad850 con 0x7f3940159ba0 2023-10-16T08:19:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39380ad9b0 con 0x7f3940159ba0 2023-10-16T08:19:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39400c7990 con 0x7f3940159ba0 2023-10-16T08:19:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380add60 con 0x7f3940159ba0 2023-10-16T08:19:30.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39400c5c70 con 0x7f3940159ba0 2023-10-16T08:19:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400c8170 con 0x7f3940159ba0 2023-10-16T08:19:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.544+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39380b4050 con 0x7f3940159ba0 2023-10-16T08:19:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.548+0000 7f393d7fa640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39240664a0 0x7f3924068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.548+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f393811ec30 con 0x7f3940159ba0 2023-10-16T08:19:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.548+0000 7f3944b47640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39240664a0 0x7f3924068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.548+0000 7f3944b47640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39240664a0 0x7f3924068960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3934007fd0 tx=0x7f3934007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.552+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39380ec4d0 con 0x7f3940159ba0 2023-10-16T08:19:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.708+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f39400cb890 con 0x7f39240664a0 2023-10-16T08:19:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.708+0000 7f393d7fa640 1 -- 172.21.15.32:0/2497786070 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3934004c80 con 0x7f39240664a0 2023-10-16T08:19:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2023-10-16T08:19:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39240664a0 msgr2=0x7f3924068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39240664a0 0x7f3924068960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3934007fd0 tx=0x7f3934007d60 comp rx=0 tx=0).stop 2023-10-16T08:19:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 msgr2=0x7f394014ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 0x7f394014ba90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f39380196c0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 shutdown_connections 2023-10-16T08:19:30.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3940159ba0 0x7f394014ba90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f39240664a0 0x7f3924068960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f39401597c0 0x7f394014a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 --2- 172.21.15.32:0/2497786070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39400c8cb0 0x7f39400c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 >> 172.21.15.32:0/2497786070 conn(0x7f394006c190 msgr2=0x7f39400cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 shutdown_connections 2023-10-16T08:19:30.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.712+0000 7f3946dab640 1 -- 172.21.15.32:0/2497786070 wait complete. 2023-10-16T08:19:30.733 INFO:tasks.ceph:Scrubbing osd.2 2023-10-16T08:19:30.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.876+0000 7fd0baf80640 1 Processor -- start 2023-10-16T08:19:30.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.876+0000 7fd0baf80640 1 -- start start 2023-10-16T08:19:30.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0baf80640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fd0b40cdc80 0x7fd0b40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0baf80640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0b415dec0 0x7fd0b40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0baf80640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b40ce460 con 0x7fd0b40d2570 2023-10-16T08:19:30.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0baf80640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b40cd830 con 0x7fd0b415dec0 2023-10-16T08:19:30.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0baf80640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b415da30 con 0x7fd0b40cdc80 2023-10-16T08:19:30.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0b415dec0 0x7fd0b40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0b415dec0 0x7fd0b40cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35200/0 (socket says 172.21.15.32:35200) 2023-10-16T08:19:30.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 -- 172.21.15.32:0/2818404887 learned_addr learned my addr 172.21.15.32:0/2818404887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b3fff640 1 --2- 172.21.15.32:0/2818404887 >> v2:172.21.15.186:3300/0 conn(0x7fd0b40cdc80 0x7fd0b40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 -- 172.21.15.32:0/2818404887 >> v2:172.21.15.186:3300/0 conn(0x7fd0b40cdc80 msgr2=0x7fd0b40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:30.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 --2- 172.21.15.32:0/2818404887 >> v2:172.21.15.186:3300/0 conn(0x7fd0b40cdc80 0x7fd0b40cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 -- 172.21.15.32:0/2818404887 >> v1:172.21.15.186:6789/0 conn(0x7fd0b40d2570 legacy=0x7fd0b4161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:30.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.880+0000 7fd0b951d640 1 -- 172.21.15.32:0/2818404887 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40ce180 con 0x7fd0b415dec0 2023-10-16T08:19:30.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0b951d640 1 --2- 172.21.15.32:0/2818404887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b415dec0 0x7fd0b40cfb50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40664e0 tx=0x7fd0ac01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4a443718652c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0b37fe640 1 -- 172.21.15.32:0/2818404887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0a5860 con 0x7fd0b415dec0 2023-10-16T08:19:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0b37fe640 1 -- 172.21.15.32:0/2818404887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac0a59c0 con 0x7fd0b415dec0 2023-10-16T08:19:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0b37fe640 1 -- 172.21.15.32:0/2818404887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0a5d50 con 0x7fd0b415dec0 2023-10-16T08:19:30.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 -- 172.21.15.32:0/2818404887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b415dec0 msgr2=0x7fd0b40cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 --2- 172.21.15.32:0/2818404887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b415dec0 0x7fd0b40cfb50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40664e0 tx=0x7fd0ac01c9a0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 -- 172.21.15.32:0/2818404887 shutdown_connections 2023-10-16T08:19:30.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 --2- 172.21.15.32:0/2818404887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b415dec0 0x7fd0b40cfb50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 --2- 172.21.15.32:0/2818404887 >> v2:172.21.15.186:3300/0 conn(0x7fd0b40cdc80 0x7fd0b40cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 -- 172.21.15.32:0/2818404887 >> 172.21.15.32:0/2818404887 conn(0x7fd0b40715c0 msgr2=0x7fd0b40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 -- 172.21.15.32:0/2818404887 shutdown_connections 2023-10-16T08:19:30.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 -- 172.21.15.32:0/2818404887 wait complete. 2023-10-16T08:19:30.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 Processor -- start 2023-10-16T08:19:30.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.884+0000 7fd0baf80640 1 -- start start 2023-10-16T08:19:30.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b40cdc80 0x7fd0b414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0b40d2570 0x7fd0b40b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0b415dec0 0x7fd0b40b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4157ab0 con 0x7fd0b40cdc80 2023-10-16T08:19:30.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4158910 con 0x7fd0b40d2570 2023-10-16T08:19:30.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4157d00 con 0x7fd0b415dec0 2023-10-16T08:19:30.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b40cdc80 0x7fd0b414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b40cdc80 0x7fd0b414ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35210/0 (socket says 172.21.15.32:35210) 2023-10-16T08:19:30.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 -- 172.21.15.32:0/3544265749 learned_addr learned my addr 172.21.15.32:0/3544265749 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b3fff640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0b40d2570 0x7fd0b40b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b951d640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0b415dec0 0x7fd0b40b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 -- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0b415dec0 msgr2=0x7fd0b40b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fd0b415dec0 0x7fd0b40b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 -- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0b40d2570 msgr2=0x7fd0b40b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0b40d2570 0x7fd0b40b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 -- 172.21.15.32:0/3544265749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009b30 con 0x7fd0b40cdc80 2023-10-16T08:19:30.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b3fff640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0b40d2570 0x7fd0b40b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:30.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b8d1c640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b40cdc80 0x7fd0b414ad00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c530 tx=0x7fd0a800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8005220 con 0x7fd0b40cdc80 2023-10-16T08:19:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8005380 con 0x7fd0b40cdc80 2023-10-16T08:19:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8007270 con 0x7fd0b40cdc80 2023-10-16T08:19:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b4157fe0 con 0x7fd0b40cdc80 2023-10-16T08:19:30.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b41585a0 con 0x7fd0b40cdc80 2023-10-16T08:19:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0b40d2950 con 0x7fd0b40cdc80 2023-10-16T08:19:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.888+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0a80073d0 con 0x7fd0b40cdc80 2023-10-16T08:19:30.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b17fa640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0980664a0 0x7fd098068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd0a8084280 con 0x7fd0b40cdc80 2023-10-16T08:19:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b17fa640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7fd09806c7a0 0x7fd09806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 --> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd09806f2b0 con 0x7fd09806c7a0 2023-10-16T08:19:30.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a80845f0 con 0x7fd0b40cdc80 2023-10-16T08:19:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b3fff640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0980664a0 0x7fd098068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b951d640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7fd09806c7a0 0x7fd09806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b951d640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7fd09806c7a0 0x7fd09806ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b3fff640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0980664a0 0x7fd098068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40cccd0 tx=0x7fd0a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.892+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== osd.2 v2:172.21.15.32:6816/1665192800 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd09806f2b0 con 0x7fd09806c7a0 2023-10-16T08:19:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 --> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd0b4158a50 con 0x7fd09806c7a0 2023-10-16T08:19:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0b17fa640 1 -- 172.21.15.32:0/3544265749 <== osd.2 v2:172.21.15.32:6816/1665192800 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd0ac009150 con 0x7fd09806c7a0 2023-10-16T08:19:30.908 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:30.909 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7fd09806c7a0 msgr2=0x7fd09806ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:6816/1665192800,v1:172.21.15.32:6817/1665192800] conn(0x7fd09806c7a0 0x7fd09806ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0980664a0 msgr2=0x7fd098068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fd0980664a0 0x7fd098068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40cccd0 tx=0x7fd0a0008040 comp rx=0 tx=0).stop 2023-10-16T08:19:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b40cdc80 msgr2=0x7fd0b414ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0baf80640 1 --2- 172.21.15.32:0/3544265749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b40cdc80 0x7fd0b414ad00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c530 tx=0x7fd0a800c9f0 comp rx=0 tx=0).stop 2023-10-16T08:19:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.904+0000 7fd0b951d640 1 -- 172.21.15.32:0/3544265749 reap_dead start 2023-10-16T08:19:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.908+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 shutdown_connections 2023-10-16T08:19:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.908+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 >> 172.21.15.32:0/3544265749 conn(0x7fd0b40715c0 msgr2=0x7fd0b4153400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.908+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 shutdown_connections 2023-10-16T08:19:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:30.908+0000 7fd0baf80640 1 -- 172.21.15.32:0/3544265749 wait complete. 2023-10-16T08:19:30.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-16T08:19:31.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d2352a640 1 Processor -- start 2023-10-16T08:19:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d2352a640 1 -- start start 2023-10-16T08:19:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d2352a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 0x7f0d1c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d2352a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c1607e0 con 0x7f0d1c15ca50 2023-10-16T08:19:31.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d2352a640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0b2b30 con 0x7f0d1c15ce30 2023-10-16T08:19:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d2352a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cb4a0 con 0x7f0d1c0b21c0 2023-10-16T08:19:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d212c6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d1c15ca50 0x7f0d1c06c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57080/0 (socket says 172.21.15.32:57080) 2023-10-16T08:19:31.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.068+0000 7f0d212c6640 1 -- 172.21.15.32:0/2883220019 learned_addr learned my addr 172.21.15.32:0/2883220019 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:31.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d21ac7640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 0x7f0d1c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181346693 0 0) 0x7f0d1c1607e0 con 0x7f0d1c15ca50 2023-10-16T08:19:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003530 con 0x7f0d1c15ca50 2023-10-16T08:19:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933040596 0 0) 0x7f0d1c0b2b30 con 0x7f0d1c15ce30 2023-10-16T08:19:31.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d1c1607e0 con 0x7f0d1c15ce30 2023-10-16T08:19:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203350957 0 0) 0x7f0d08003530 con 0x7f0d1c15ca50 2023-10-16T08:19:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 msgr2=0x7f0d1c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 0x7f0d1c0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 >> v1:172.21.15.186:6789/0 conn(0x7f0d1c15ce30 legacy=0x7f0d1c0cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0cf060 con 0x7f0d1c15ca50 2023-10-16T08:19:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7f0d18001680 con 0x7f0d1c15ca50 2023-10-16T08:19:31.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 >> v1:172.21.15.32:6789/0 conn(0x7f0d1c15ca50 legacy=0x7f0d1c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d08003b40 0x7f0d08003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d08004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 0x7f0d08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cb4a0 con 0x7f0d080047d0 2023-10-16T08:19:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800ba40 con 0x7f0d08003b40 2023-10-16T08:19:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800bbd0 con 0x7f0d1c0b21c0 2023-10-16T08:19:31.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d08004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d212c6640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d08003b40 0x7f0d08003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d21ac7640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 0x7f0d08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d20ac5640 1 -- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 msgr2=0x7f0d08005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c0b21c0 0x7f0d08005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d20ac5640 1 -- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d08003b40 msgr2=0x7f0d08003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d08003b40 0x7f0d08003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.072+0000 7f0d20ac5640 1 -- 172.21.15.32:0/2883220019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08003530 con 0x7f0d080047d0 2023-10-16T08:19:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d08004c50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b2cc0 tx=0x7f0d0c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47a2e6b5115860 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0052d0 con 0x7f0d080047d0 2023-10-16T08:19:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c005430 con 0x7f0d080047d0 2023-10-16T08:19:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d13fff640 1 -- 172.21.15.32:0/2883220019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0a8870 con 0x7f0d080047d0 2023-10-16T08:19:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- 172.21.15.32:0/2883220019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 msgr2=0x7f0d08004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2883220019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d08004c50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b2cc0 tx=0x7f0d0c01ec50 comp rx=0 tx=0).stop 2023-10-16T08:19:31.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d21ac7640 1 -- 172.21.15.32:0/2883220019 reap_dead start 2023-10-16T08:19:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- 172.21.15.32:0/2883220019 shutdown_connections 2023-10-16T08:19:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- 172.21.15.32:0/2883220019 >> 172.21.15.32:0/2883220019 conn(0x7f0d1c0cd1c0 msgr2=0x7f0d1c0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- 172.21.15.32:0/2883220019 shutdown_connections 2023-10-16T08:19:31.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- 172.21.15.32:0/2883220019 wait complete. 2023-10-16T08:19:31.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 Processor -- start 2023-10-16T08:19:31.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- start start 2023-10-16T08:19:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d1c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d1c14edc0 0x7f0d1c1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c1e82d0 0x7f0d1c0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0c9310 con 0x7f0d080047d0 2023-10-16T08:19:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c1e8750 con 0x7f0d1c14edc0 2023-10-16T08:19:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.076+0000 7f0d2352a640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0c8690 con 0x7f0d1c1e82d0 2023-10-16T08:19:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d1c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d1c0c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35230/0 (socket says 172.21.15.32:35230) 2023-10-16T08:19:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 -- 172.21.15.32:0/2467134138 learned_addr learned my addr 172.21.15.32:0/2467134138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d1c14edc0 0x7f0d1c1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d21ac7640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c1e82d0 0x7f0d1c0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 -- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c1e82d0 msgr2=0x7f0d1c0c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c1e82d0 0x7f0d1c0c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 -- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d1c14edc0 msgr2=0x7f0d1c1e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d1c14edc0 0x7f0d1c1e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 -- 172.21.15.32:0/2467134138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0c8970 con 0x7f0d080047d0 2023-10-16T08:19:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d212c6640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d1c0c9730 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0d18004f50 tx=0x7f0d180074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18007740 con 0x7f0d080047d0 2023-10-16T08:19:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1800f940 con 0x7f0d080047d0 2023-10-16T08:19:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800fcd0 con 0x7f0d080047d0 2023-10-16T08:19:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c0c8c50 con 0x7f0d080047d0 2023-10-16T08:19:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c14f220 con 0x7f0d080047d0 2023-10-16T08:19:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c074010 con 0x7f0d080047d0 2023-10-16T08:19:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0d1801d460 con 0x7f0d080047d0 2023-10-16T08:19:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.080+0000 7f0d11ffb640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d000664a0 0x7f0d00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.084+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0d18083e20 con 0x7f0d080047d0 2023-10-16T08:19:31.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.084+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d000664a0 0x7f0d00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.084+0000 7f0d20ac5640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d000664a0 0x7f0d00068960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b0000 tx=0x7f0d0c001370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.088+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0d18051460 con 0x7f0d080047d0 2023-10-16T08:19:31.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.244+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d1c0704f0 con 0x7f0d000664a0 2023-10-16T08:19:31.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.244+0000 7f0d11ffb640 1 -- 172.21.15.32:0/2467134138 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0d0c005290 con 0x7f0d000664a0 2023-10-16T08:19:31.247 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2023-10-16T08:19:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d000664a0 msgr2=0x7f0d00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d000664a0 0x7f0d00068960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b0000 tx=0x7f0d0c001370 comp rx=0 tx=0).stop 2023-10-16T08:19:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 msgr2=0x7f0d1c0c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d1c0c9730 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0d18004f50 tx=0x7f0d180074f0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 shutdown_connections 2023-10-16T08:19:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0d000664a0 0x7f0d00068960 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0d1c1e82d0 0x7f0d1c0c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d1c14edc0 0x7f0d1c1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 --2- 172.21.15.32:0/2467134138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080047d0 0x7f0d1c0c9730 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 >> 172.21.15.32:0/2467134138 conn(0x7f0d1c0cd1c0 msgr2=0x7f0d1c0b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 shutdown_connections 2023-10-16T08:19:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.248+0000 7f0d2352a640 1 -- 172.21.15.32:0/2467134138 wait complete. 2023-10-16T08:19:31.268 INFO:tasks.ceph:Scrubbing osd.3 2023-10-16T08:19:31.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 Processor -- start 2023-10-16T08:19:31.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 -- start start 2023-10-16T08:19:31.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe8741441a0 0x7fe8740b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8740cb530 0x7fe8740c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87415e5f0 con 0x7fe8740b1a90 2023-10-16T08:19:31.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740b1f20 con 0x7fe8740cb530 2023-10-16T08:19:31.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.412+0000 7fe879c54640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87415e8d0 con 0x7fe8741441a0 2023-10-16T08:19:31.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe872ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8740b1a90 0x7fe8740cae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57096/0 (socket says 172.21.15.32:57096) 2023-10-16T08:19:31.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe872ffd640 1 -- 172.21.15.32:0/515141034 learned_addr learned my addr 172.21.15.32:0/515141034 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe873fff640 1 --2- 172.21.15.32:0/515141034 >> v2:172.21.15.32:3300/0 conn(0x7fe8740cb530 0x7fe8740c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe8737fe640 1 --2- 172.21.15.32:0/515141034 >> v2:172.21.15.186:3300/0 conn(0x7fe8741441a0 0x7fe8740b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe8727fc640 1 -- 172.21.15.32:0/515141034 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965973367 0 0) 0x7fe87415e5f0 con 0x7fe8740b1a90 2023-10-16T08:19:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe8727fc640 1 -- 172.21.15.32:0/515141034 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c003530 con 0x7fe8740b1a90 2023-10-16T08:19:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe873fff640 1 -- 172.21.15.32:0/515141034 >> v2:172.21.15.186:3300/0 conn(0x7fe8741441a0 msgr2=0x7fe8740b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe873fff640 1 --2- 172.21.15.32:0/515141034 >> v2:172.21.15.186:3300/0 conn(0x7fe8741441a0 0x7fe8740b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe873fff640 1 -- 172.21.15.32:0/515141034 >> v1:172.21.15.32:6789/0 conn(0x7fe8740b1a90 legacy=0x7fe8740cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe873fff640 1 -- 172.21.15.32:0/515141034 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87415ebb0 con 0x7fe8740cb530 2023-10-16T08:19:31.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe873fff640 1 --2- 172.21.15.32:0/515141034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740cb530 0x7fe8740c8fc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe8740cc0c0 tx=0x7fe86c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30593ecca4ef8181 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe8727fc640 1 -- 172.21.15.32:0/515141034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0a57d0 con 0x7fe8740cb530 2023-10-16T08:19:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe8727fc640 1 -- 172.21.15.32:0/515141034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c0a5930 con 0x7fe8740cb530 2023-10-16T08:19:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.416+0000 7fe8727fc640 1 -- 172.21.15.32:0/515141034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0a5cc0 con 0x7fe8740cb530 2023-10-16T08:19:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- 172.21.15.32:0/515141034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740cb530 msgr2=0x7fe8740c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 --2- 172.21.15.32:0/515141034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740cb530 0x7fe8740c8fc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe8740cc0c0 tx=0x7fe86c01c910 comp rx=0 tx=0).stop 2023-10-16T08:19:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- 172.21.15.32:0/515141034 shutdown_connections 2023-10-16T08:19:31.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 --2- 172.21.15.32:0/515141034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740cb530 0x7fe8740c8fc0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 --2- 172.21.15.32:0/515141034 >> v2:172.21.15.186:3300/0 conn(0x7fe8741441a0 0x7fe8740b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- 172.21.15.32:0/515141034 >> 172.21.15.32:0/515141034 conn(0x7fe87406a820 msgr2=0x7fe8740b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- 172.21.15.32:0/515141034 shutdown_connections 2023-10-16T08:19:31.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- 172.21.15.32:0/515141034 wait complete. 2023-10-16T08:19:31.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 Processor -- start 2023-10-16T08:19:31.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- start start 2023-10-16T08:19:31.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740b1a90 0x7fe8741f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8740cb530 0x7fe87413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8741441a0 0x7fe874142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87413f100 con 0x7fe8740b1a90 2023-10-16T08:19:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87413f3e0 con 0x7fe8740cb530 2023-10-16T08:19:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe879c54640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741f8590 con 0x7fe8741441a0 2023-10-16T08:19:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740b1a90 0x7fe8741f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740b1a90 0x7fe8741f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35260/0 (socket says 172.21.15.32:35260) 2023-10-16T08:19:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 -- 172.21.15.32:0/2991397603 learned_addr learned my addr 172.21.15.32:0/2991397603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe872ffd640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8740cb530 0x7fe87413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe873fff640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8741441a0 0x7fe874142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 -- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8741441a0 msgr2=0x7fe874142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8741441a0 0x7fe874142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 -- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8740cb530 msgr2=0x7fe87413e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8740cb530 0x7fe87413e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe8737fe640 1 -- 172.21.15.32:0/2991397603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c009a70 con 0x7fe8740b1a90 2023-10-16T08:19:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe872ffd640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8740cb530 0x7fe87413e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.420+0000 7fe873fff640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe8741441a0 0x7fe874142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe8737fe640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740b1a90 0x7fe8741f8d90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe86400e870 tx=0x7fe86400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864009510 con 0x7fe8740b1a90 2023-10-16T08:19:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86400eec0 con 0x7fe8740b1a90 2023-10-16T08:19:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86400cb40 con 0x7fe8740b1a90 2023-10-16T08:19:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8741f8870 con 0x7fe8740b1a90 2023-10-16T08:19:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87413f6c0 con 0x7fe8740b1a90 2023-10-16T08:19:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe87404e170 con 0x7fe8740b1a90 2023-10-16T08:19:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe86400cca0 con 0x7fe8740b1a90 2023-10-16T08:19:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8600664a0 0x7fe860068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe864083e00 con 0x7fe8740b1a90 2023-10-16T08:19:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7fe86006c7a0 0x7fe86006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 --> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe86006f2b0 con 0x7fe86006c7a0 2023-10-16T08:19:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe864086380 con 0x7fe8740b1a90 2023-10-16T08:19:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe872ffd640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8600664a0 0x7fe860068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.424+0000 7fe873fff640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7fe86006c7a0 0x7fe86006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.428+0000 7fe872ffd640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8600664a0 0x7fe860068960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe8740b1710 tx=0x7fe868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.428+0000 7fe873fff640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7fe86006c7a0 0x7fe86006ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.428+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== osd.3 v2:172.21.15.32:6824/2556335256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe86006f2b0 con 0x7fe86006c7a0 2023-10-16T08:19:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 --> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe87406b4d0 con 0x7fe86006c7a0 2023-10-16T08:19:31.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe85bfff640 1 -- 172.21.15.32:0/2991397603 <== osd.3 v2:172.21.15.32:6824/2556335256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe87406b4d0 con 0x7fe86006c7a0 2023-10-16T08:19:31.445 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:31.446 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:31.446 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:31.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7fe86006c7a0 msgr2=0x7fe86006ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:6824/2556335256,v1:172.21.15.32:6825/2556335256] conn(0x7fe86006c7a0 0x7fe86006ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8600664a0 msgr2=0x7fe860068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe8600664a0 0x7fe860068960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe8740b1710 tx=0x7fe868009040 comp rx=0 tx=0).stop 2023-10-16T08:19:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740b1a90 msgr2=0x7fe8741f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 --2- 172.21.15.32:0/2991397603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8740b1a90 0x7fe8741f8d90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe86400e870 tx=0x7fe86400ec70 comp rx=0 tx=0).stop 2023-10-16T08:19:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe873fff640 1 -- 172.21.15.32:0/2991397603 reap_dead start 2023-10-16T08:19:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 shutdown_connections 2023-10-16T08:19:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 >> 172.21.15.32:0/2991397603 conn(0x7fe87406a820 msgr2=0x7fe874071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 shutdown_connections 2023-10-16T08:19:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.440+0000 7fe879c54640 1 -- 172.21.15.32:0/2991397603 wait complete. 2023-10-16T08:19:31.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-16T08:19:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.604+0000 7f0e40b35640 1 Processor -- start 2023-10-16T08:19:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.604+0000 7f0e40b35640 1 -- start start 2023-10-16T08:19:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e40b35640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 0x7f0e3c0d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e40b35640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f0e3c0d3550 0x7f0e3c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e40b35640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c0c65f0 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e40b35640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c0ce820 con 0x7f0e3c0d3550 2023-10-16T08:19:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e40b35640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0cfa40 con 0x7f0e3c15ca70 2023-10-16T08:19:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 0x7f0e3c0d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 0x7f0e3c0d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:50512/0 (socket says 172.21.15.32:50512) 2023-10-16T08:19:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 -- 172.21.15.32:0/4256056706 learned_addr learned my addr 172.21.15.32:0/4256056706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e33fff640 1 --2- 172.21.15.32:0/4256056706 >> v2:172.21.15.186:3300/0 conn(0x7f0e3c0d3550 0x7f0e3c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 -- 172.21.15.32:0/4256056706 >> v2:172.21.15.186:3300/0 conn(0x7f0e3c0d3550 msgr2=0x7f0e3c0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 --2- 172.21.15.32:0/4256056706 >> v2:172.21.15.186:3300/0 conn(0x7f0e3c0d3550 0x7f0e3c0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 -- 172.21.15.32:0/4256056706 >> v1:172.21.15.186:6789/0 conn(0x7f0e3c0cf5d0 legacy=0x7f0e3c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.608+0000 7f0e3a59c640 1 -- 172.21.15.32:0/4256056706 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c0cdf80 con 0x7f0e3c15ca70 2023-10-16T08:19:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e3a59c640 1 --2- 172.21.15.32:0/4256056706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 0x7f0e3c0d3010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c006fd0 tx=0x7f0e2c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eecd0c55f67ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e39d9b640 1 -- 172.21.15.32:0/4256056706 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0aa650 con 0x7f0e3c15ca70 2023-10-16T08:19:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e39d9b640 1 -- 172.21.15.32:0/4256056706 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c0aa7b0 con 0x7f0e3c15ca70 2023-10-16T08:19:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e39d9b640 1 -- 172.21.15.32:0/4256056706 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0aab40 con 0x7f0e3c15ca70 2023-10-16T08:19:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- 172.21.15.32:0/4256056706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 msgr2=0x7f0e3c0d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 --2- 172.21.15.32:0/4256056706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 0x7f0e3c0d3010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c006fd0 tx=0x7f0e2c01c4f0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- 172.21.15.32:0/4256056706 shutdown_connections 2023-10-16T08:19:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 --2- 172.21.15.32:0/4256056706 >> v2:172.21.15.186:3300/0 conn(0x7f0e3c0d3550 0x7f0e3c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 --2- 172.21.15.32:0/4256056706 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c15ca70 0x7f0e3c0d3010 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- 172.21.15.32:0/4256056706 >> 172.21.15.32:0/4256056706 conn(0x7f0e3c09bc50 msgr2=0x7f0e3c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- 172.21.15.32:0/4256056706 shutdown_connections 2023-10-16T08:19:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- 172.21.15.32:0/4256056706 wait complete. 2023-10-16T08:19:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 Processor -- start 2023-10-16T08:19:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- start start 2023-10-16T08:19:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 0x7f0e3c154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3c0d3550 0x7f0e3c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c15ca70 0x7f0e3c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c15ea90 con 0x7f0e3c0d3550 2023-10-16T08:19:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c15ed70 con 0x7f0e3c15ca70 2023-10-16T08:19:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.612+0000 7f0e40b35640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c15f050 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 0x7f0e3c154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3c0d3550 0x7f0e3c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 0x7f0e3c154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:50528/0 (socket says 172.21.15.32:50528) 2023-10-16T08:19:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 -- 172.21.15.32:0/704856866 learned_addr learned my addr 172.21.15.32:0/704856866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3ad9d640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c15ca70 0x7f0e3c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 -- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c15ca70 msgr2=0x7f0e3c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c15ca70 0x7f0e3c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 -- 172.21.15.32:0/704856866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3c0d3550 msgr2=0x7f0e3c0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3c0d3550 0x7f0e3c0b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 -- 172.21.15.32:0/704856866 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c009480 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e33fff640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3c0d3550 0x7f0e3c0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:19:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e3a59c640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 0x7f0e3c154f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c01a120 tx=0x7f0e2c019290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c005ba0 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c0b8070 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c15f330 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0aa920 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c155480 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c0cfda0 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.616+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mon.2 v2:172.21.15.186:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0e2c0aaa80 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.620+0000 7f0e32ffd640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e1c0664a0 0x7f0e1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.620+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mon.2 v2:172.21.15.186:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0e2c0ba070 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.620+0000 7f0e33fff640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e1c0664a0 0x7f0e1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.620+0000 7f0e33fff640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e1c0664a0 0x7f0e1c068960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0cc9f0 tx=0x7f0e24009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.624+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mon.2 v2:172.21.15.186:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0e2c0ebef0 con 0x7f0e3c0cf5d0 2023-10-16T08:19:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.780+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e3c074010 con 0x7f0e1c0664a0 2023-10-16T08:19:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.780+0000 7f0e32ffd640 1 -- 172.21.15.32:0/704856866 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0e3c074010 con 0x7f0e1c0664a0 2023-10-16T08:19:31.784 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2023-10-16T08:19:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e1c0664a0 msgr2=0x7f0e1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e1c0664a0 0x7f0e1c068960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0cc9f0 tx=0x7f0e24009190 comp rx=0 tx=0).stop 2023-10-16T08:19:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 msgr2=0x7f0e3c154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 0x7f0e3c154f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c01a120 tx=0x7f0e2c019290 comp rx=0 tx=0).stop 2023-10-16T08:19:31.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 shutdown_connections 2023-10-16T08:19:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e3c15ca70 0x7f0e3c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3c0d3550 0x7f0e3c0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f0e1c0664a0 0x7f0e1c068960 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 --2- 172.21.15.32:0/704856866 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f0e3c0cf5d0 0x7f0e3c154f40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 >> 172.21.15.32:0/704856866 conn(0x7f0e3c09bc50 msgr2=0x7f0e3c159250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 shutdown_connections 2023-10-16T08:19:31.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.784+0000 7f0e40b35640 1 -- 172.21.15.32:0/704856866 wait complete. 2023-10-16T08:19:31.806 INFO:tasks.ceph:Scrubbing osd.4 2023-10-16T08:19:31.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 Processor -- start 2023-10-16T08:19:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 -- start start 2023-10-16T08:19:31.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c1595a0 0x7fbd3c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fbd3c159980 0x7fbd3c06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd3c0b3bc0 0x7fbd3c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c0b45b0 con 0x7fbd3c0b3bc0 2023-10-16T08:19:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c0c8a20 con 0x7fbd3c159980 2023-10-16T08:19:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4238d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0c8c10 con 0x7fbd3c1595a0 2023-10-16T08:19:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd3c0b3bc0 0x7fbd3c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd3c0b3bc0 0x7fbd3c0b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35280/0 (socket says 172.21.15.32:35280) 2023-10-16T08:19:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 -- 172.21.15.32:0/1740856251 learned_addr learned my addr 172.21.15.32:0/1740856251 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd3bfff640 1 --2- 172.21.15.32:0/1740856251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c1595a0 0x7fbd3c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd3b7fe640 1 --2- 172.21.15.32:0/1740856251 >> v2:172.21.15.186:3300/0 conn(0x7fbd3c159980 0x7fbd3c06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 -- 172.21.15.32:0/1740856251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c1595a0 msgr2=0x7fbd3c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 --2- 172.21.15.32:0/1740856251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c1595a0 0x7fbd3c06c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 -- 172.21.15.32:0/1740856251 >> v2:172.21.15.186:3300/0 conn(0x7fbd3c159980 msgr2=0x7fbd3c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 --2- 172.21.15.32:0/1740856251 >> v2:172.21.15.186:3300/0 conn(0x7fbd3c159980 0x7fbd3c06c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.952+0000 7fbd4092a640 1 -- 172.21.15.32:0/1740856251 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0c8ef0 con 0x7fbd3c0b3bc0 2023-10-16T08:19:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4092a640 1 --2- 172.21.15.32:0/1740856251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c0b3bc0 0x7fbd3c0b4070 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c06e4f0 tx=0x7fbd3401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0ce98e4341e8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd3affd640 1 -- 172.21.15.32:0/1740856251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a5790 con 0x7fbd3c0b3bc0 2023-10-16T08:19:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd3affd640 1 -- 172.21.15.32:0/1740856251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340a58f0 con 0x7fbd3c0b3bc0 2023-10-16T08:19:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd3affd640 1 -- 172.21.15.32:0/1740856251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a5c80 con 0x7fbd3c0b3bc0 2023-10-16T08:19:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 -- 172.21.15.32:0/1740856251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c0b3bc0 msgr2=0x7fbd3c0b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 --2- 172.21.15.32:0/1740856251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c0b3bc0 0x7fbd3c0b4070 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c06e4f0 tx=0x7fbd3401c910 comp rx=0 tx=0).stop 2023-10-16T08:19:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 -- 172.21.15.32:0/1740856251 shutdown_connections 2023-10-16T08:19:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 --2- 172.21.15.32:0/1740856251 >> v2:172.21.15.186:3300/0 conn(0x7fbd3c159980 0x7fbd3c06c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 --2- 172.21.15.32:0/1740856251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c0b3bc0 0x7fbd3c0b4070 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 --2- 172.21.15.32:0/1740856251 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c1595a0 0x7fbd3c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 -- 172.21.15.32:0/1740856251 >> 172.21.15.32:0/1740856251 conn(0x7fbd3c06a820 msgr2=0x7fbd3c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 -- 172.21.15.32:0/1740856251 shutdown_connections 2023-10-16T08:19:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.956+0000 7fbd4238d640 1 -- 172.21.15.32:0/1740856251 wait complete. 2023-10-16T08:19:31.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 Processor -- start 2023-10-16T08:19:31.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- start start 2023-10-16T08:19:31.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c0b3bc0 0x7fbd3c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3c1595a0 0x7fbd3c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c070400 0x7fbd3c13cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c156ba0 con 0x7fbd3c070400 2023-10-16T08:19:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c15c000 con 0x7fbd3c1595a0 2023-10-16T08:19:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c15c2b0 con 0x7fbd3c0b3bc0 2023-10-16T08:19:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c070400 0x7fbd3c13cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c070400 0x7fbd3c13cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35284/0 (socket says 172.21.15.32:35284) 2023-10-16T08:19:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 -- 172.21.15.32:0/4028763971 learned_addr learned my addr 172.21.15.32:0/4028763971 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd3b7fe640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3c1595a0 0x7fbd3c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd3bfff640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c0b3bc0 0x7fbd3c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 -- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c0b3bc0 msgr2=0x7fbd3c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c0b3bc0 0x7fbd3c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 -- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3c1595a0 msgr2=0x7fbd3c15bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd3c1595a0 0x7fbd3c15bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 -- 172.21.15.32:0/4028763971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009b30 con 0x7fbd3c070400 2023-10-16T08:19:31.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd3bfff640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fbd3c0b3bc0 0x7fbd3c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:31.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4092a640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c070400 0x7fbd3c13cbc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbd340af040 tx=0x7fbd340196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b8670 con 0x7fbd3c070400 2023-10-16T08:19:31.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340b87d0 con 0x7fbd3c070400 2023-10-16T08:19:31.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c15c590 con 0x7fbd3c070400 2023-10-16T08:19:31.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b8b60 con 0x7fbd3c070400 2023-10-16T08:19:31.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c157090 con 0x7fbd3c070400 2023-10-16T08:19:31.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.960+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd3c15c7d0 con 0x7fbd3c070400 2023-10-16T08:19:31.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbd340b8cc0 con 0x7fbd3c070400 2023-10-16T08:19:31.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd38ff9640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd280664a0 0x7fbd28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fbd3411f400 con 0x7fbd3c070400 2023-10-16T08:19:31.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd38ff9640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbd2806c7a0 0x7fbd2806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:31.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 --> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd2806f2b0 con 0x7fbd2806c7a0 2023-10-16T08:19:31.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd341219e0 con 0x7fbd3c070400 2023-10-16T08:19:31.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd3bfff640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd280664a0 0x7fbd28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd3b7fe640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbd2806c7a0 0x7fbd2806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:31.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd3bfff640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd280664a0 0x7fbd28068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0097c0 tx=0x7fbd2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.964+0000 7fbd3b7fe640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbd2806c7a0 0x7fbd2806ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:31.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.968+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== osd.4 v2:172.21.15.186:6824/2991882832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fbd2806f2b0 con 0x7fbd2806c7a0 2023-10-16T08:19:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 --> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbd3c0c9790 con 0x7fbd2806c7a0 2023-10-16T08:19:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd38ff9640 1 -- 172.21.15.32:0/4028763971 <== osd.4 v2:172.21.15.186:6824/2991882832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbd3000ba60 con 0x7fbd2806c7a0 2023-10-16T08:19:31.982 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:31.983 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbd2806c7a0 msgr2=0x7fbd2806ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6824/2991882832,v1:172.21.15.186:6825/2991882832] conn(0x7fbd2806c7a0 0x7fbd2806ebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd280664a0 msgr2=0x7fbd28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fbd280664a0 0x7fbd28068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0097c0 tx=0x7fbd2c008040 comp rx=0 tx=0).stop 2023-10-16T08:19:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c070400 msgr2=0x7fbd3c13cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 --2- 172.21.15.32:0/4028763971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd3c070400 0x7fbd3c13cbc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbd340af040 tx=0x7fbd340196b0 comp rx=0 tx=0).stop 2023-10-16T08:19:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4092a640 1 -- 172.21.15.32:0/4028763971 reap_dead start 2023-10-16T08:19:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 shutdown_connections 2023-10-16T08:19:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 >> 172.21.15.32:0/4028763971 conn(0x7fbd3c06a820 msgr2=0x7fbd3c0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 shutdown_connections 2023-10-16T08:19:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:31.980+0000 7fbd4238d640 1 -- 172.21.15.32:0/4028763971 wait complete. 2023-10-16T08:19:32.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-16T08:19:32.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.140+0000 7f037baa9640 1 Processor -- start 2023-10-16T08:19:32.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.140+0000 7f037baa9640 1 -- start start 2023-10-16T08:19:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f037baa9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f037415ca50 0x7f037406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f037baa9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f037415ce30 0x7f03740cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f037baa9640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03741607e0 con 0x7f03740b21c0 2023-10-16T08:19:32.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f037baa9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03740cf060 con 0x7f037415ca50 2023-10-16T08:19:32.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f037baa9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0374070210 con 0x7f037415ce30 2023-10-16T08:19:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f037415ca50 0x7f037406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f037415ca50 0x7f037406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35294/0 (socket says 172.21.15.32:35294) 2023-10-16T08:19:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 -- 172.21.15.32:0/596450758 learned_addr learned my addr 172.21.15.32:0/596450758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379044640 1 --2- 172.21.15.32:0/596450758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f037415ce30 0x7f03740cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 -- 172.21.15.32:0/596450758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f037415ce30 msgr2=0x7f03740cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 --2- 172.21.15.32:0/596450758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f037415ce30 0x7f03740cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 -- 172.21.15.32:0/596450758 >> v1:172.21.15.186:6789/0 conn(0x7f03740b21c0 legacy=0x7f0374160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.144+0000 7f0379845640 1 -- 172.21.15.32:0/596450758 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740704f0 con 0x7f037415ca50 2023-10-16T08:19:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f0379845640 1 --2- 172.21.15.32:0/596450758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f037406c980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0370009870 tx=0x7f037001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b7b1b6d50b5d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f0378843640 1 -- 172.21.15.32:0/596450758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700ab620 con 0x7f037415ca50 2023-10-16T08:19:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f0378843640 1 -- 172.21.15.32:0/596450758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03700ab780 con 0x7f037415ca50 2023-10-16T08:19:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f0378843640 1 -- 172.21.15.32:0/596450758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700abb10 con 0x7f037415ca50 2023-10-16T08:19:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 -- 172.21.15.32:0/596450758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 msgr2=0x7f037406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 --2- 172.21.15.32:0/596450758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f037406c980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0370009870 tx=0x7f037001c510 comp rx=0 tx=0).stop 2023-10-16T08:19:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 -- 172.21.15.32:0/596450758 shutdown_connections 2023-10-16T08:19:32.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 --2- 172.21.15.32:0/596450758 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f037415ce30 0x7f03740cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 --2- 172.21.15.32:0/596450758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f037406c980 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 -- 172.21.15.32:0/596450758 >> 172.21.15.32:0/596450758 conn(0x7f03740cd1c0 msgr2=0x7f03740cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:32.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 -- 172.21.15.32:0/596450758 shutdown_connections 2023-10-16T08:19:32.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 -- 172.21.15.32:0/596450758 wait complete. 2023-10-16T08:19:32.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 Processor -- start 2023-10-16T08:19:32.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.148+0000 7f037baa9640 1 -- start start 2023-10-16T08:19:32.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03740b21c0 0x7f037406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f0374151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f037415ce30 0x7f0374152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037406c6f0 con 0x7f037415ca50 2023-10-16T08:19:32.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0374154550 con 0x7f037415ce30 2023-10-16T08:19:32.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0374153c20 con 0x7f03740b21c0 2023-10-16T08:19:32.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f0374151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f0374151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35302/0 (socket says 172.21.15.32:35302) 2023-10-16T08:19:32.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 -- 172.21.15.32:0/798034771 learned_addr learned my addr 172.21.15.32:0/798034771 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:32.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379845640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03740b21c0 0x7f037406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037a046640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f037415ce30 0x7f0374152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 -- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03740b21c0 msgr2=0x7f037406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03740b21c0 0x7f037406bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 -- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f037415ce30 msgr2=0x7f0374152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f037415ce30 0x7f0374152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 -- 172.21.15.32:0/798034771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03700094d0 con 0x7f037415ca50 2023-10-16T08:19:32.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379845640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03740b21c0 0x7f037406bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:32.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037a046640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f037415ce30 0x7f0374152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f0379044640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f0374151f80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f036400e870 tx=0x7f036400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364009510 con 0x7f037415ca50 2023-10-16T08:19:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036400eec0 con 0x7f037415ca50 2023-10-16T08:19:32.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400cb40 con 0x7f037415ca50 2023-10-16T08:19:32.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03740b2b60 con 0x7f037415ca50 2023-10-16T08:19:32.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03741541e0 con 0x7f037415ca50 2023-10-16T08:19:32.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0374074010 con 0x7f037415ca50 2023-10-16T08:19:32.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.152+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f036400ccc0 con 0x7f037415ca50 2023-10-16T08:19:32.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.156+0000 7f036a7fc640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03580664f0 0x7f03580689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.156+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0364084150 con 0x7f037415ca50 2023-10-16T08:19:32.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.156+0000 7f0379845640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03580664f0 0x7f03580689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.156+0000 7f0379845640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03580664f0 0x7f03580689b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f03700af000 tx=0x7f0370008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.160+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0364051770 con 0x7f037415ca50 2023-10-16T08:19:32.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.312+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f03740b29e0 con 0x7f03580664f0 2023-10-16T08:19:32.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f036a7fc640 1 -- 172.21.15.32:0/798034771 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03700bb090 con 0x7f03580664f0 2023-10-16T08:19:32.317 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2023-10-16T08:19:32.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03580664f0 msgr2=0x7f03580689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03580664f0 0x7f03580689b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f03700af000 tx=0x7f0370008040 comp rx=0 tx=0).stop 2023-10-16T08:19:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 msgr2=0x7f0374151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f0374151f80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f036400e870 tx=0x7f036400ec70 comp rx=0 tx=0).stop 2023-10-16T08:19:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 shutdown_connections 2023-10-16T08:19:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f037415ce30 0x7f0374152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f037415ca50 0x7f0374151f80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f03580664f0 0x7f03580689b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 --2- 172.21.15.32:0/798034771 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f03740b21c0 0x7f037406bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 >> 172.21.15.32:0/798034771 conn(0x7f03740cd1c0 msgr2=0x7f0374159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 shutdown_connections 2023-10-16T08:19:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.316+0000 7f037baa9640 1 -- 172.21.15.32:0/798034771 wait complete. 2023-10-16T08:19:32.338 INFO:tasks.ceph:Scrubbing osd.5 2023-10-16T08:19:32.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 Processor -- start 2023-10-16T08:19:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 -- start start 2023-10-16T08:19:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b6c1595a0 0x7f8b6c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0b42d0 0x7f8b6c06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6c0c8a20 con 0x7f8b6c159980 2023-10-16T08:19:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c15e300 con 0x7f8b6c1595a0 2023-10-16T08:19:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b7373e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c8c10 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b714da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b6c1595a0 0x7f8b6c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b714da640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b6c1595a0 0x7f8b6c06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35310/0 (socket says 172.21.15.32:35310) 2023-10-16T08:19:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.484+0000 7f8b714da640 1 -- 172.21.15.32:0/248387192 learned_addr learned my addr 172.21.15.32:0/248387192 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b71cdb640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0b42d0 0x7f8b6c06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b714da640 1 -- 172.21.15.32:0/248387192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0b42d0 msgr2=0x7f8b6c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b714da640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0b42d0 0x7f8b6c06c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b714da640 1 -- 172.21.15.32:0/248387192 >> v1:172.21.15.186:6789/0 conn(0x7f8b6c159980 legacy=0x7f8b6c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b714da640 1 -- 172.21.15.32:0/248387192 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c0c8ef0 con 0x7f8b6c1595a0 2023-10-16T08:19:32.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b71cdb640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0b42d0 0x7f8b6c06c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b714da640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c1595a0 0x7f8b6c06c4b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009870 tx=0x7f8b6801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d73d37288d9040 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b63fff640 1 -- 172.21.15.32:0/248387192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680ab680 con 0x7f8b6c1595a0 2023-10-16T08:19:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b63fff640 1 -- 172.21.15.32:0/248387192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b680ab7e0 con 0x7f8b6c1595a0 2023-10-16T08:19:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b63fff640 1 -- 172.21.15.32:0/248387192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680abb70 con 0x7f8b6c1595a0 2023-10-16T08:19:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b7373e640 1 -- 172.21.15.32:0/248387192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c1595a0 msgr2=0x7f8b6c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.488+0000 7f8b7373e640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c1595a0 0x7f8b6c06c4b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009870 tx=0x7f8b6801c480 comp rx=0 tx=0).stop 2023-10-16T08:19:32.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- 172.21.15.32:0/248387192 shutdown_connections 2023-10-16T08:19:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0b42d0 0x7f8b6c06c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 --2- 172.21.15.32:0/248387192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c1595a0 0x7f8b6c06c4b0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- 172.21.15.32:0/248387192 >> 172.21.15.32:0/248387192 conn(0x7f8b6c06a820 msgr2=0x7f8b6c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- 172.21.15.32:0/248387192 shutdown_connections 2023-10-16T08:19:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- 172.21.15.32:0/248387192 wait complete. 2023-10-16T08:19:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 Processor -- start 2023-10-16T08:19:32.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- start start 2023-10-16T08:19:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c0b42d0 0x7f8b6c14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b6c1595a0 0x7f8b6c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0cb510 0x7f8b6c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0cbf70 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0cb9b0 con 0x7f8b6c1595a0 2023-10-16T08:19:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b7373e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0cbc90 con 0x7f8b6c0cb510 2023-10-16T08:19:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c0b42d0 0x7f8b6c14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c0b42d0 0x7f8b6c14e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35312/0 (socket says 172.21.15.32:35312) 2023-10-16T08:19:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 -- 172.21.15.32:0/353114982 learned_addr learned my addr 172.21.15.32:0/353114982 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b71cdb640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0cb510 0x7f8b6c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b70cd9640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b6c1595a0 0x7f8b6c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 -- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0cb510 msgr2=0x7f8b6c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f8b6c0cb510 0x7f8b6c155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 -- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b6c1595a0 msgr2=0x7f8b6c0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b6c1595a0 0x7f8b6c0cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.492+0000 7f8b714da640 1 -- 172.21.15.32:0/353114982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680094d0 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b714da640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c0b42d0 0x7f8b6c14e3f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ad040 tx=0x7f8b6801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680b48c0 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b680b4a40 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c159ea0 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680b4df0 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c14e930 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b6c0c9790 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8b680c7440 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8b500664a0 0x7f8b50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f8b6811ed10 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7f8b5006c7a0 0x7f8b5006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 --> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b5006f2b0 con 0x7f8b5006c7a0 2023-10-16T08:19:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b681212f0 con 0x7f8b6c0b42d0 2023-10-16T08:19:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b70cd9640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8b500664a0 0x7f8b50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.496+0000 7f8b71cdb640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7f8b5006c7a0 0x7f8b5006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.500+0000 7f8b70cd9640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8b500664a0 0x7f8b50068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0cca20 tx=0x7f8b5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.500+0000 7f8b71cdb640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7f8b5006c7a0 0x7f8b5006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.500+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== osd.5 v2:172.21.15.186:6801/3150472125 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8b5006f2b0 con 0x7f8b5006c7a0 2023-10-16T08:19:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 --> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8b6c04e170 con 0x7f8b5006c7a0 2023-10-16T08:19:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b61ffb640 1 -- 172.21.15.32:0/353114982 <== osd.5 v2:172.21.15.186:6801/3150472125 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8b6c04e170 con 0x7f8b5006c7a0 2023-10-16T08:19:32.516 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:32.517 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7f8b5006c7a0 msgr2=0x7f8b5006ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6801/3150472125,v1:172.21.15.186:6803/3150472125] conn(0x7f8b5006c7a0 0x7f8b5006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8b500664a0 msgr2=0x7f8b50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f8b500664a0 0x7f8b50068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0cca20 tx=0x7f8b5c009040 comp rx=0 tx=0).stop 2023-10-16T08:19:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c0b42d0 msgr2=0x7f8b6c14e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 --2- 172.21.15.32:0/353114982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b6c0b42d0 0x7f8b6c14e3f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ad040 tx=0x7f8b6801acc0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b71cdb640 1 -- 172.21.15.32:0/353114982 reap_dead start 2023-10-16T08:19:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 shutdown_connections 2023-10-16T08:19:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 >> 172.21.15.32:0/353114982 conn(0x7f8b6c06a820 msgr2=0x7f8b6c0b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 shutdown_connections 2023-10-16T08:19:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.512+0000 7f8b7373e640 1 -- 172.21.15.32:0/353114982 wait complete. 2023-10-16T08:19:32.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-16T08:19:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f848b194640 1 Processor -- start 2023-10-16T08:19:32.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f848b194640 1 -- start start 2023-10-16T08:19:32.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f848b194640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84840ae240 0x7f84841625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f848b194640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848415af10 con 0x7f84840cdcc0 2023-10-16T08:19:32.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f848b194640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848406ac80 con 0x7f848415ab30 2023-10-16T08:19:32.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f848b194640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848406b890 con 0x7f84840ae240 2023-10-16T08:19:32.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f8483fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84840ae240 0x7f84841625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f8489731640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84840cdcc0 0x7f8484164b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57110/0 (socket says 172.21.15.32:57110) 2023-10-16T08:19:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f8489731640 1 -- 172.21.15.32:0/165243257 learned_addr learned my addr 172.21.15.32:0/165243257 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f8483fff640 1 -- 172.21.15.32:0/165243257 >> v1:172.21.15.186:6789/0 conn(0x7f848415ab30 legacy=0x7f84840ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f8483fff640 1 -- 172.21.15.32:0/165243257 >> v1:172.21.15.32:6789/0 conn(0x7f84840cdcc0 legacy=0x7f8484164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.676+0000 7f8483fff640 1 -- 172.21.15.32:0/165243257 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848406af60 con 0x7f84840ae240 2023-10-16T08:19:32.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f8483fff640 1 --2- 172.21.15.32:0/165243257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84841625c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f84700097c0 tx=0x7f847001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e87110bf0ed389 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f84837fe640 1 -- 172.21.15.32:0/165243257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ab650 con 0x7f84840ae240 2023-10-16T08:19:32.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f84837fe640 1 -- 172.21.15.32:0/165243257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700ab7b0 con 0x7f84840ae240 2023-10-16T08:19:32.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f84837fe640 1 -- 172.21.15.32:0/165243257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700abb40 con 0x7f84840ae240 2023-10-16T08:19:32.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 -- 172.21.15.32:0/165243257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 msgr2=0x7f84841625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 --2- 172.21.15.32:0/165243257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84841625c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f84700097c0 tx=0x7f847001c450 comp rx=0 tx=0).stop 2023-10-16T08:19:32.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 -- 172.21.15.32:0/165243257 shutdown_connections 2023-10-16T08:19:32.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 --2- 172.21.15.32:0/165243257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84841625c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 -- 172.21.15.32:0/165243257 >> 172.21.15.32:0/165243257 conn(0x7f84840b0880 msgr2=0x7f84840b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:32.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 -- 172.21.15.32:0/165243257 shutdown_connections 2023-10-16T08:19:32.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.680+0000 7f848b194640 1 -- 172.21.15.32:0/165243257 wait complete. 2023-10-16T08:19:32.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 Processor -- start 2023-10-16T08:19:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 -- start start 2023-10-16T08:19:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84840b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84840cdcc0 0x7f848409eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f848415ab30 0x7f848406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848415bd80 con 0x7f84840ae240 2023-10-16T08:19:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848415b140 con 0x7f84840cdcc0 2023-10-16T08:19:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848415b3e0 con 0x7f848415ab30 2023-10-16T08:19:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84840b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84840b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35334/0 (socket says 172.21.15.32:35334) 2023-10-16T08:19:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 -- 172.21.15.32:0/440678088 learned_addr learned my addr 172.21.15.32:0/440678088 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8483fff640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84840cdcc0 0x7f848409eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8489731640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f848415ab30 0x7f848406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 -- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f848415ab30 msgr2=0x7f848406d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f848415ab30 0x7f848406d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 -- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84840cdcc0 msgr2=0x7f848409eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84840cdcc0 0x7f848409eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 -- 172.21.15.32:0/440678088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009420 con 0x7f84840ae240 2023-10-16T08:19:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8489731640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f848415ab30 0x7f848406d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f8488f30640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84840b1ab0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f847800c3c0 tx=0x7f847800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780052d0 con 0x7f84840ae240 2023-10-16T08:19:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478005430 con 0x7f84840ae240 2023-10-16T08:19:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478003840 con 0x7f84840ae240 2023-10-16T08:19:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848415b6c0 con 0x7f84840ae240 2023-10-16T08:19:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.684+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848409f0a0 con 0x7f84840ae240 2023-10-16T08:19:32.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.688+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8484074010 con 0x7f84840ae240 2023-10-16T08:19:32.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.688+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f84780039a0 con 0x7f84840ae240 2023-10-16T08:19:32.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.688+0000 7f84817fa640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f84680664a0 0x7f8468068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.688+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f8478083140 con 0x7f84840ae240 2023-10-16T08:19:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.688+0000 7f8483fff640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f84680664a0 0x7f8468068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.692+0000 7f8483fff640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f84680664a0 0x7f8468068960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84700af000 tx=0x7f84700a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:32.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.692+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8478050820 con 0x7f84840ae240 2023-10-16T08:19:32.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8484168a50 con 0x7f84680664a0 2023-10-16T08:19:32.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f84817fa640 1 -- 172.21.15.32:0/440678088 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f84700aba20 con 0x7f84680664a0 2023-10-16T08:19:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2023-10-16T08:19:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f84680664a0 msgr2=0x7f8468068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f84680664a0 0x7f8468068960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84700af000 tx=0x7f84700a7000 comp rx=0 tx=0).stop 2023-10-16T08:19:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 msgr2=0x7f84840b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84840b1ab0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f847800c3c0 tx=0x7f847800c880 comp rx=0 tx=0).stop 2023-10-16T08:19:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 shutdown_connections 2023-10-16T08:19:32.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f84680664a0 0x7f8468068960 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f848415ab30 0x7f848406d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84840cdcc0 0x7f848409eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 --2- 172.21.15.32:0/440678088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84840ae240 0x7f84840b1ab0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 >> 172.21.15.32:0/440678088 conn(0x7f84840b0880 msgr2=0x7f8484069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:32.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 shutdown_connections 2023-10-16T08:19:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:32.856+0000 7f848b194640 1 -- 172.21.15.32:0/440678088 wait complete. 2023-10-16T08:19:32.880 INFO:tasks.ceph:Scrubbing osd.6 2023-10-16T08:19:32.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.024+0000 7f2d9466b640 1 Processor -- start 2023-10-16T08:19:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.024+0000 7f2d9466b640 1 -- start start 2023-10-16T08:19:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d9466b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c1595a0 0x7f2d8c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d9466b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d8c0b42d0 0x7f2d8c06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d9466b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0c8a20 con 0x7f2d8c159980 2023-10-16T08:19:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d9466b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c15e300 con 0x7f2d8c0b42d0 2023-10-16T08:19:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d9466b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0c8c10 con 0x7f2d8c1595a0 2023-10-16T08:19:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d91c06640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d8c159980 0x7f2d8c0b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47286/0 (socket says 172.21.15.32:47286) 2023-10-16T08:19:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d91c06640 1 -- 172.21.15.32:0/1627659963 learned_addr learned my addr 172.21.15.32:0/1627659963 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92c08640 1 --2- 172.21.15.32:0/1627659963 >> v2:172.21.15.32:3300/0 conn(0x7f2d8c0b42d0 0x7f2d8c06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92407640 1 --2- 172.21.15.32:0/1627659963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c1595a0 0x7f2d8c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92c08640 1 -- 172.21.15.32:0/1627659963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c1595a0 msgr2=0x7f2d8c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92c08640 1 --2- 172.21.15.32:0/1627659963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c1595a0 0x7f2d8c06c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92c08640 1 -- 172.21.15.32:0/1627659963 >> v1:172.21.15.32:6789/0 conn(0x7f2d8c159980 legacy=0x7f2d8c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92c08640 1 -- 172.21.15.32:0/1627659963 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0c8ef0 con 0x7f2d8c0b42d0 2023-10-16T08:19:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d91405640 1 -- 172.21.15.32:0/1627659963 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350490278 0 0) 0x7f2d8c0c8a20 con 0x7f2d8c159980 2023-10-16T08:19:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.028+0000 7f2d92c08640 1 --2- 172.21.15.32:0/1627659963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c0b42d0 0x7f2d8c06c9f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009d90 tx=0x7f2d8401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbf7bd149b8c086 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d91405640 1 -- 172.21.15.32:0/1627659963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840a5950 con 0x7f2d8c0b42d0 2023-10-16T08:19:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d91405640 1 -- 172.21.15.32:0/1627659963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d840a5ab0 con 0x7f2d8c0b42d0 2023-10-16T08:19:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d91405640 1 -- 172.21.15.32:0/1627659963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840a5e40 con 0x7f2d8c0b42d0 2023-10-16T08:19:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- 172.21.15.32:0/1627659963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c0b42d0 msgr2=0x7f2d8c06c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 --2- 172.21.15.32:0/1627659963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c0b42d0 0x7f2d8c06c9f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009d90 tx=0x7f2d8401c890 comp rx=0 tx=0).stop 2023-10-16T08:19:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- 172.21.15.32:0/1627659963 shutdown_connections 2023-10-16T08:19:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 --2- 172.21.15.32:0/1627659963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c0b42d0 0x7f2d8c06c9f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 --2- 172.21.15.32:0/1627659963 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c1595a0 0x7f2d8c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- 172.21.15.32:0/1627659963 >> 172.21.15.32:0/1627659963 conn(0x7f2d8c06a820 msgr2=0x7f2d8c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- 172.21.15.32:0/1627659963 shutdown_connections 2023-10-16T08:19:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- 172.21.15.32:0/1627659963 wait complete. 2023-10-16T08:19:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 Processor -- start 2023-10-16T08:19:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- start start 2023-10-16T08:19:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c0b42d0 0x7f2d8c1f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d8c1595a0 0x7f2d8c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c15a3d0 0x7f2d8c152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c154380 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c15a860 con 0x7f2d8c1595a0 2023-10-16T08:19:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.032+0000 7f2d9466b640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1540a0 con 0x7f2d8c0b42d0 2023-10-16T08:19:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c15a3d0 0x7f2d8c152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c15a3d0 0x7f2d8c152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50542/0 (socket says 172.21.15.32:50542) 2023-10-16T08:19:33.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 -- 172.21.15.32:0/3730823653 learned_addr learned my addr 172.21.15.32:0/3730823653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92407640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c0b42d0 0x7f2d8c1f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d91c06640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d8c1595a0 0x7f2d8c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 -- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c0b42d0 msgr2=0x7f2d8c1f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f2d8c0b42d0 0x7f2d8c1f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 -- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d8c1595a0 msgr2=0x7f2d8c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d8c1595a0 0x7f2d8c159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 -- 172.21.15.32:0/3730823653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009a70 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d91c06640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d8c1595a0 0x7f2d8c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:33.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d92c08640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c15a3d0 0x7f2d8c152a10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2d8401af40 tx=0x7f2d84019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840ade40 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84009450 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840b9670 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c153770 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c153d30 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d8c0c9790 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.036+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2d840b97d0 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d82ffd640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d700664a0 0x7f2d70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2d8411f590 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d82ffd640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f2d7006c7a0 0x7f2d7006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 --> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d7006f2b0 con 0x7f2d7006c7a0 2023-10-16T08:19:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d84121b70 con 0x7f2d8c15a3d0 2023-10-16T08:19:33.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d92407640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d700664a0 0x7f2d70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d91c06640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f2d7006c7a0 0x7f2d7006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d91c06640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f2d7006c7a0 0x7f2d7006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d92407640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d700664a0 0x7f2d70068960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2d88007a60 tx=0x7f2d88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.040+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== osd.6 v2:172.21.15.186:6804/1135936083 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f2d7006f2b0 con 0x7f2d7006c7a0 2023-10-16T08:19:33.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.052+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 --> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2d8c1541e0 con 0x7f2d7006c7a0 2023-10-16T08:19:33.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.052+0000 7f2d82ffd640 1 -- 172.21.15.32:0/3730823653 <== osd.6 v2:172.21.15.186:6804/1135936083 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2d8c1541e0 con 0x7f2d7006c7a0 2023-10-16T08:19:33.057 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:33.057 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:33.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f2d7006c7a0 msgr2=0x7f2d7006ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6804/1135936083,v1:172.21.15.186:6808/1135936083] conn(0x7f2d7006c7a0 0x7f2d7006ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d700664a0 msgr2=0x7f2d70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f2d700664a0 0x7f2d70068960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2d88007a60 tx=0x7f2d88008040 comp rx=0 tx=0).stop 2023-10-16T08:19:33.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c15a3d0 msgr2=0x7f2d8c152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 --2- 172.21.15.32:0/3730823653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d8c15a3d0 0x7f2d8c152a10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2d8401af40 tx=0x7f2d84019630 comp rx=0 tx=0).stop 2023-10-16T08:19:33.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d92c08640 1 -- 172.21.15.32:0/3730823653 reap_dead start 2023-10-16T08:19:33.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 shutdown_connections 2023-10-16T08:19:33.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 >> 172.21.15.32:0/3730823653 conn(0x7f2d8c06a820 msgr2=0x7f2d8c1520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 shutdown_connections 2023-10-16T08:19:33.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.056+0000 7f2d9466b640 1 -- 172.21.15.32:0/3730823653 wait complete. 2023-10-16T08:19:33.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-16T08:19:33.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 Processor -- start 2023-10-16T08:19:33.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 -- start start 2023-10-16T08:19:33.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2000d15d0 0x7fe2000cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d3010 0x7fe2000c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000ce770 con 0x7fe20015cb90 2023-10-16T08:19:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000d0600 con 0x7fe2000d15d0 2023-10-16T08:19:33.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.216+0000 7fe207172640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c6450 con 0x7fe2000d3010 2023-10-16T08:19:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2000d15d0 0x7fe2000cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2000d15d0 0x7fe2000cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50558/0 (socket says 172.21.15.32:50558) 2023-10-16T08:19:33.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 -- 172.21.15.32:0/706627903 learned_addr learned my addr 172.21.15.32:0/706627903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe20570f640 1 --2- 172.21.15.32:0/706627903 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d3010 0x7fe2000c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 -- 172.21.15.32:0/706627903 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d3010 msgr2=0x7fe2000c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:33.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 --2- 172.21.15.32:0/706627903 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d3010 0x7fe2000c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 -- 172.21.15.32:0/706627903 >> v1:172.21.15.186:6789/0 conn(0x7fe20015cb90 legacy=0x7fe2000d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 -- 172.21.15.32:0/706627903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000d08a0 con 0x7fe2000d15d0 2023-10-16T08:19:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1fffff640 1 --2- 172.21.15.32:0/706627903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2000d15d0 0x7fe2000cfb10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009870 tx=0x7fe1ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd251066241f06ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1ff7fe640 1 -- 172.21.15.32:0/706627903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0ab680 con 0x7fe2000d15d0 2023-10-16T08:19:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1ff7fe640 1 -- 172.21.15.32:0/706627903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec0ab7e0 con 0x7fe2000d15d0 2023-10-16T08:19:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.220+0000 7fe1ff7fe640 1 -- 172.21.15.32:0/706627903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0abb70 con 0x7fe2000d15d0 2023-10-16T08:19:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- 172.21.15.32:0/706627903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2000d15d0 msgr2=0x7fe2000cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 --2- 172.21.15.32:0/706627903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2000d15d0 0x7fe2000cfb10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009870 tx=0x7fe1ec01c480 comp rx=0 tx=0).stop 2023-10-16T08:19:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- 172.21.15.32:0/706627903 shutdown_connections 2023-10-16T08:19:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 --2- 172.21.15.32:0/706627903 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d3010 0x7fe2000c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 --2- 172.21.15.32:0/706627903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2000d15d0 0x7fe2000cfb10 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- 172.21.15.32:0/706627903 >> 172.21.15.32:0/706627903 conn(0x7fe20009bc50 msgr2=0x7fe2000d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- 172.21.15.32:0/706627903 shutdown_connections 2023-10-16T08:19:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- 172.21.15.32:0/706627903 wait complete. 2023-10-16T08:19:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 Processor -- start 2023-10-16T08:19:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- start start 2023-10-16T08:19:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d15d0 0x7fe200151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2000d3010 0x7fe2000c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 0x7fe20006b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200151e70 con 0x7fe20015cb90 2023-10-16T08:19:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200153910 con 0x7fe2000d3010 2023-10-16T08:19:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe207172640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001f7d70 con 0x7fe2000d15d0 2023-10-16T08:19:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 0x7fe20006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 0x7fe20006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50572/0 (socket says 172.21.15.32:50572) 2023-10-16T08:19:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 -- 172.21.15.32:0/1685095531 learned_addr learned my addr 172.21.15.32:0/1685095531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe1fffff640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2000d3010 0x7fe2000c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe204f0e640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d15d0 0x7fe200151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 -- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d15d0 msgr2=0x7fe200151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d15d0 0x7fe200151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 -- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2000d3010 msgr2=0x7fe2000c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2000d3010 0x7fe2000c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.224+0000 7fe20570f640 1 -- 172.21.15.32:0/1685095531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec0094d0 con 0x7fe20015cb90 2023-10-16T08:19:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe204f0e640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d15d0 0x7fe200151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe20570f640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 0x7fe20006b110 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800efc0 tx=0x7fe1f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8014480 con 0x7fe20015cb90 2023-10-16T08:19:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800f040 con 0x7fe20015cb90 2023-10-16T08:19:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8014820 con 0x7fe20015cb90 2023-10-16T08:19:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2001f8010 con 0x7fe20015cb90 2023-10-16T08:19:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2001f85d0 con 0x7fe20015cb90 2023-10-16T08:19:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200074010 con 0x7fe20015cb90 2023-10-16T08:19:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe1f8014980 con 0x7fe20015cb90 2023-10-16T08:19:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe1fd7fa640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1e40664d0 0x7fe1e4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe1f80847b0 con 0x7fe20015cb90 2023-10-16T08:19:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.228+0000 7fe204f0e640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1e40664d0 0x7fe1e4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.232+0000 7fe204f0e640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1e40664d0 0x7fe1e4068990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4007fd0 tx=0x7fe1f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.232+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe1f8051de0 con 0x7fe20015cb90 2023-10-16T08:19:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.392+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe20015cf70 con 0x7fe1e40664d0 2023-10-16T08:19:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.392+0000 7fe1fd7fa640 1 -- 172.21.15.32:0/1685095531 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe1f4001cf0 con 0x7fe1e40664d0 2023-10-16T08:19:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2023-10-16T08:19:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.392+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1e40664d0 msgr2=0x7fe1e4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.392+0000 7fe207172640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1e40664d0 0x7fe1e4068990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4007fd0 tx=0x7fe1f4009040 comp rx=0 tx=0).stop 2023-10-16T08:19:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.392+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 msgr2=0x7fe20006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.392+0000 7fe207172640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 0x7fe20006b110 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800efc0 tx=0x7fe1f800c490 comp rx=0 tx=0).stop 2023-10-16T08:19:33.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 shutdown_connections 2023-10-16T08:19:33.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe20015cb90 0x7fe20006b110 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2000d3010 0x7fe2000c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe1e40664d0 0x7fe1e4068990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 --2- 172.21.15.32:0/1685095531 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe2000d15d0 0x7fe200151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 >> 172.21.15.32:0/1685095531 conn(0x7fe20009bc50 msgr2=0x7fe2000d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 shutdown_connections 2023-10-16T08:19:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.396+0000 7fe207172640 1 -- 172.21.15.32:0/1685095531 wait complete. 2023-10-16T08:19:33.416 INFO:tasks.ceph:Scrubbing osd.7 2023-10-16T08:19:33.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:19:33.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.560+0000 7f01a971d640 1 Processor -- start 2023-10-16T08:19:33.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.560+0000 7f01a971d640 1 -- start start 2023-10-16T08:19:33.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.560+0000 7f01a971d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a41441a0 0x7f01a40b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.560+0000 7f01a971d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40b1a90 0x7f01a40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.560+0000 7f01a971d640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f01a40cb360 0x7f01a406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a971d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a40b1ef0 con 0x7f01a41441a0 2023-10-16T08:19:33.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a971d640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a415e5f0 con 0x7f01a40cb360 2023-10-16T08:19:33.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a971d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a415e8d0 con 0x7f01a40b1a90 2023-10-16T08:19:33.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a41441a0 0x7f01a40b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a41441a0 0x7f01a40b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50588/0 (socket says 172.21.15.32:50588) 2023-10-16T08:19:33.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 -- 172.21.15.32:0/2145550874 learned_addr learned my addr 172.21.15.32:0/2145550874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 -- 172.21.15.32:0/2145550874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40b1a90 msgr2=0x7f01a40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:33.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 --2- 172.21.15.32:0/2145550874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40b1a90 0x7f01a40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 -- 172.21.15.32:0/2145550874 >> v2:172.21.15.186:3300/0 conn(0x7f01a40cb360 msgr2=0x7f01a406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:33.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 --2- 172.21.15.32:0/2145550874 >> v2:172.21.15.186:3300/0 conn(0x7f01a40cb360 0x7f01a406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 -- 172.21.15.32:0/2145550874 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a415ebb0 con 0x7f01a41441a0 2023-10-16T08:19:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a2ffd640 1 --2- 172.21.15.32:0/2145550874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a41441a0 0x7f01a40b1550 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f01a406d670 tx=0x7f019401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afedacb57a31752 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a27fc640 1 -- 172.21.15.32:0/2145550874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940ab600 con 0x7f01a41441a0 2023-10-16T08:19:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a27fc640 1 -- 172.21.15.32:0/2145550874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01940a7050 con 0x7f01a41441a0 2023-10-16T08:19:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.564+0000 7f01a27fc640 1 -- 172.21.15.32:0/2145550874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940ab980 con 0x7f01a41441a0 2023-10-16T08:19:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- 172.21.15.32:0/2145550874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a41441a0 msgr2=0x7f01a40b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- 172.21.15.32:0/2145550874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a41441a0 0x7f01a40b1550 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f01a406d670 tx=0x7f019401c240 comp rx=0 tx=0).stop 2023-10-16T08:19:33.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- 172.21.15.32:0/2145550874 shutdown_connections 2023-10-16T08:19:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- 172.21.15.32:0/2145550874 >> v2:172.21.15.186:3300/0 conn(0x7f01a40cb360 0x7f01a406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- 172.21.15.32:0/2145550874 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40b1a90 0x7f01a40cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- 172.21.15.32:0/2145550874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a41441a0 0x7f01a40b1550 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- 172.21.15.32:0/2145550874 >> 172.21.15.32:0/2145550874 conn(0x7f01a406a820 msgr2=0x7f01a40b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- 172.21.15.32:0/2145550874 shutdown_connections 2023-10-16T08:19:33.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- 172.21.15.32:0/2145550874 wait complete. 2023-10-16T08:19:33.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 Processor -- start 2023-10-16T08:19:33.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- start start 2023-10-16T08:19:33.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a40b1a90 0x7f01a4142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40cb360 0x7f01a413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41441a0 0x7f01a41f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4140440 con 0x7f01a40b1a90 2023-10-16T08:19:33.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a41f95b0 con 0x7f01a41441a0 2023-10-16T08:19:33.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a971d640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4141d70 con 0x7f01a40cb360 2023-10-16T08:19:33.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a40b1a90 0x7f01a4142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a40b1a90 0x7f01a4142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50592/0 (socket says 172.21.15.32:50592) 2023-10-16T08:19:33.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.568+0000 7f01a2ffd640 1 -- 172.21.15.32:0/4034159525 learned_addr learned my addr 172.21.15.32:0/4034159525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a37fe640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41441a0 0x7f01a41f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a2ffd640 1 -- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40cb360 msgr2=0x7f01a413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019bfff640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40cb360 0x7f01a413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a2ffd640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40cb360 0x7f01a413ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a2ffd640 1 -- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41441a0 msgr2=0x7f01a41f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a2ffd640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41441a0 0x7f01a41f9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a2ffd640 1 -- 172.21.15.32:0/4034159525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940094d0 con 0x7f01a40b1a90 2023-10-16T08:19:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a37fe640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a41441a0 0x7f01a41f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019bfff640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f01a40cb360 0x7f01a413ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a2ffd640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a40b1a90 0x7f01a4142ad0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f019401cae0 tx=0x7f019401aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940b6620 con 0x7f01a40b1a90 2023-10-16T08:19:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01940b67a0 con 0x7f01a40b1a90 2023-10-16T08:19:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01940b6b50 con 0x7f01a40b1a90 2023-10-16T08:19:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a4140630 con 0x7f01a40b1a90 2023-10-16T08:19:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a41422d0 con 0x7f01a40b1a90 2023-10-16T08:19:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01a40cb7d0 con 0x7f01a40b1a90 2023-10-16T08:19:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f01940b2040 con 0x7f01a40b1a90 2023-10-16T08:19:33.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f01840664a0 0x7f0184068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f019411e250 con 0x7f01a40b1a90 2023-10-16T08:19:33.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7f018406c7a0 0x7f018406ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 --> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018406f2b0 con 0x7f018406c7a0 2023-10-16T08:19:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.572+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0194120830 con 0x7f01a40b1a90 2023-10-16T08:19:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.576+0000 7f019bfff640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f01840664a0 0x7f0184068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.576+0000 7f01a37fe640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7f018406c7a0 0x7f018406ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.576+0000 7f019bfff640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f01840664a0 0x7f0184068960 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01a40ca7c0 tx=0x7f018c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.576+0000 7f01a37fe640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7f018406c7a0 0x7f018406ebe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.576+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== osd.7 v2:172.21.15.186:6800/403837937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f018406f2b0 con 0x7f018406c7a0 2023-10-16T08:19:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 --> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f01a404e170 con 0x7f018406c7a0 2023-10-16T08:19:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f019b7fe640 1 -- 172.21.15.32:0/4034159525 <== osd.7 v2:172.21.15.186:6800/403837937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f01a404e170 con 0x7f018406c7a0 2023-10-16T08:19:33.592 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-10-16T08:19:33.592 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:19:33.593 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-10-16T08:19:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7f018406c7a0 msgr2=0x7f018406ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6800/403837937,v1:172.21.15.186:6802/403837937] conn(0x7f018406c7a0 0x7f018406ebe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f01840664a0 msgr2=0x7f0184068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f01840664a0 0x7f0184068960 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01a40ca7c0 tx=0x7f018c008040 comp rx=0 tx=0).stop 2023-10-16T08:19:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a40b1a90 msgr2=0x7f01a4142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 --2- 172.21.15.32:0/4034159525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a40b1a90 0x7f01a4142ad0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f019401cae0 tx=0x7f019401aa80 comp rx=0 tx=0).stop 2023-10-16T08:19:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a37fe640 1 -- 172.21.15.32:0/4034159525 reap_dead start 2023-10-16T08:19:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 shutdown_connections 2023-10-16T08:19:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 >> 172.21.15.32:0/4034159525 conn(0x7f01a406a820 msgr2=0x7f01a4071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 shutdown_connections 2023-10-16T08:19:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.588+0000 7f01a971d640 1 -- 172.21.15.32:0/4034159525 wait complete. 2023-10-16T08:19:33.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-16T08:19:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 Processor -- start 2023-10-16T08:19:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 -- start start 2023-10-16T08:19:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5160070c40 0x7f516015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7f51600b07f0 0x7f5160164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516015be60 con 0x7f51600b4020 2023-10-16T08:19:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516015b3b0 con 0x7f5160070c40 2023-10-16T08:19:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f5166101640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516015c1f0 con 0x7f51600b07f0 2023-10-16T08:19:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5160070c40 0x7f516015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5160070c40 0x7f516015ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50600/0 (socket says 172.21.15.32:50600) 2023-10-16T08:19:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 -- 172.21.15.32:0/863717252 learned_addr learned my addr 172.21.15.32:0/863717252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515ffff640 1 --2- 172.21.15.32:0/863717252 >> v2:172.21.15.186:3300/0 conn(0x7f51600b07f0 0x7f5160164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 -- 172.21.15.32:0/863717252 >> v2:172.21.15.186:3300/0 conn(0x7f51600b07f0 msgr2=0x7f5160164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 --2- 172.21.15.32:0/863717252 >> v2:172.21.15.186:3300/0 conn(0x7f51600b07f0 0x7f5160164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 -- 172.21.15.32:0/863717252 >> v1:172.21.15.186:6789/0 conn(0x7f51600b4020 legacy=0x7f51600b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515f7fe640 1 -- 172.21.15.32:0/863717252 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160071020 con 0x7f5160070c40 2023-10-16T08:19:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.752+0000 7f515ffff640 1 --2- 172.21.15.32:0/863717252 >> v2:172.21.15.186:3300/0 conn(0x7f51600b07f0 0x7f5160164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f515f7fe640 1 --2- 172.21.15.32:0/863717252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160070c40 0x7f516015ae70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f516006e590 tx=0x7f515001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502cf6732f1d6f14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f515e7fc640 1 -- 172.21.15.32:0/863717252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51500a7070 con 0x7f5160070c40 2023-10-16T08:19:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f515e7fc640 1 -- 172.21.15.32:0/863717252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51500ab430 con 0x7f5160070c40 2023-10-16T08:19:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f515e7fc640 1 -- 172.21.15.32:0/863717252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51500ab7c0 con 0x7f5160070c40 2023-10-16T08:19:33.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 -- 172.21.15.32:0/863717252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160070c40 msgr2=0x7f516015ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 --2- 172.21.15.32:0/863717252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160070c40 0x7f516015ae70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f516006e590 tx=0x7f515001c430 comp rx=0 tx=0).stop 2023-10-16T08:19:33.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 -- 172.21.15.32:0/863717252 shutdown_connections 2023-10-16T08:19:33.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 --2- 172.21.15.32:0/863717252 >> v2:172.21.15.186:3300/0 conn(0x7f51600b07f0 0x7f5160164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 --2- 172.21.15.32:0/863717252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160070c40 0x7f516015ae70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 -- 172.21.15.32:0/863717252 >> 172.21.15.32:0/863717252 conn(0x7f516006c190 msgr2=0x7f516006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 -- 172.21.15.32:0/863717252 shutdown_connections 2023-10-16T08:19:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.756+0000 7f5166101640 1 -- 172.21.15.32:0/863717252 wait complete. 2023-10-16T08:19:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 Processor -- start 2023-10-16T08:19:33.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 -- start start 2023-10-16T08:19:33.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160070c40 0x7f51600b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51600b07f0 0x7f51600b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 0x7f516014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51600b1510 con 0x7f51600b4020 2023-10-16T08:19:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51600b17f0 con 0x7f5160070c40 2023-10-16T08:19:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51600b1ad0 con 0x7f51600b07f0 2023-10-16T08:19:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 0x7f516014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 0x7f516014d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50604/0 (socket says 172.21.15.32:50604) 2023-10-16T08:19:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 -- 172.21.15.32:0/4090497252 learned_addr learned my addr 172.21.15.32:0/4090497252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515f7fe640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160070c40 0x7f51600b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515effd640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51600b07f0 0x7f51600b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 -- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51600b07f0 msgr2=0x7f51600b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51600b07f0 0x7f51600b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 -- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160070c40 msgr2=0x7f51600b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160070c40 0x7f51600b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 -- 172.21.15.32:0/4090497252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51500094d0 con 0x7f51600b4020 2023-10-16T08:19:33.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f515ffff640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 0x7f516014d990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f515800ce40 tx=0x7f515800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800d840 con 0x7f51600b4020 2023-10-16T08:19:33.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515800d9a0 con 0x7f51600b4020 2023-10-16T08:19:33.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51600b1db0 con 0x7f51600b4020 2023-10-16T08:19:33.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800dd50 con 0x7f51600b4020 2023-10-16T08:19:33.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.760+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51600aef10 con 0x7f51600b4020 2023-10-16T08:19:33.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.764+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5160074010 con 0x7f51600b4020 2023-10-16T08:19:33.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.764+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f515801e3b0 con 0x7f51600b4020 2023-10-16T08:19:33.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.764+0000 7f5147fff640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f514c0664a0 0x7f514c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:33.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.764+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f51580846d0 con 0x7f51600b4020 2023-10-16T08:19:33.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.764+0000 7f515f7fe640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f514c0664a0 0x7f514c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:33.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.764+0000 7f515f7fe640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f514c0664a0 0x7f514c068960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f51500af000 tx=0x7f5150008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:33.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.768+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5158051d20 con 0x7f51600b4020 2023-10-16T08:19:33.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f516015bb80 con 0x7f514c0664a0 2023-10-16T08:19:33.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5147fff640 1 -- 172.21.15.32:0/4090497252 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f51500bc090 con 0x7f514c0664a0 2023-10-16T08:19:33.931 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2023-10-16T08:19:33.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f514c0664a0 msgr2=0x7f514c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f514c0664a0 0x7f514c068960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f51500af000 tx=0x7f5150008040 comp rx=0 tx=0).stop 2023-10-16T08:19:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 msgr2=0x7f516014d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 0x7f516014d990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f515800ce40 tx=0x7f515800a710 comp rx=0 tx=0).stop 2023-10-16T08:19:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 shutdown_connections 2023-10-16T08:19:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51600b4020 0x7f516014d990 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f514c0664a0 0x7f514c068960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f51600b07f0 0x7f51600b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 --2- 172.21.15.32:0/4090497252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5160070c40 0x7f51600b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:33.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.928+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 >> 172.21.15.32:0/4090497252 conn(0x7f516006c190 msgr2=0x7f51600af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:33.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.932+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 shutdown_connections 2023-10-16T08:19:33.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:33.932+0000 7f5166101640 1 -- 172.21.15.32:0/4090497252 wait complete. 2023-10-16T08:19:33.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:19:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.092+0000 7fe306b72640 1 Processor -- start 2023-10-16T08:19:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.092+0000 7fe306b72640 1 -- start start 2023-10-16T08:19:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe306b72640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 0x7fe30015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe306b72640 1 --2- >> v2:172.21.15.186:3300/0 conn(0x7fe30015d690 0x7fe300164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe306b72640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30015be60 con 0x7fe300070c40 2023-10-16T08:19:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe306b72640 1 -- --> v2:172.21.15.186:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30015b3b0 con 0x7fe30015d690 2023-10-16T08:19:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe306b72640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30015c1f0 con 0x7fe3000b07f0 2023-10-16T08:19:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 0x7fe30015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 0x7fe30015ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3301/0 says I am v2:172.21.15.32:50716/0 (socket says 172.21.15.32:50716) 2023-10-16T08:19:34.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 -- 172.21.15.32:0/2457564266 learned_addr learned my addr 172.21.15.32:0/2457564266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:34.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe30490e640 1 --1- 172.21.15.32:0/2457564266 >> v1:172.21.15.186:6789/0 conn(0x7fe300070c40 0x7fe3000b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.186:6789/0 says I am v1:172.21.15.32:39782/0 (socket says 172.21.15.32:39782) 2023-10-16T08:19:34.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe30510f640 1 --2- 172.21.15.32:0/2457564266 >> v2:172.21.15.186:3300/0 conn(0x7fe30015d690 0x7fe300164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:34.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 -- 172.21.15.32:0/2457564266 >> v2:172.21.15.186:3300/0 conn(0x7fe30015d690 msgr2=0x7fe300164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 --2- 172.21.15.32:0/2457564266 >> v2:172.21.15.186:3300/0 conn(0x7fe30015d690 0x7fe300164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 -- 172.21.15.32:0/2457564266 >> v1:172.21.15.186:6789/0 conn(0x7fe300070c40 legacy=0x7fe3000b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2fffff640 1 -- 172.21.15.32:0/2457564266 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300071020 con 0x7fe3000b07f0 2023-10-16T08:19:34.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.096+0000 7fe2ff7fe640 1 -- 172.21.15.32:0/2457564266 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76840619 0 0) 0x7fe30015be60 con 0x7fe300070c40 2023-10-16T08:19:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe2fffff640 1 --2- 172.21.15.32:0/2457564266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 0x7fe30015ae70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009870 tx=0x7fe2ec01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e19a8c374c78599 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe2ff7fe640 1 -- 172.21.15.32:0/2457564266 <== mon.2 v2:172.21.15.186:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0aa610 con 0x7fe3000b07f0 2023-10-16T08:19:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe2ff7fe640 1 -- 172.21.15.32:0/2457564266 <== mon.2 v2:172.21.15.186:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0a6040 con 0x7fe3000b07f0 2023-10-16T08:19:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe2ff7fe640 1 -- 172.21.15.32:0/2457564266 <== mon.2 v2:172.21.15.186:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0aa990 con 0x7fe3000b07f0 2023-10-16T08:19:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- 172.21.15.32:0/2457564266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 msgr2=0x7fe30015ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 --2- 172.21.15.32:0/2457564266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 0x7fe30015ae70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009870 tx=0x7fe2ec01c4b0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- 172.21.15.32:0/2457564266 shutdown_connections 2023-10-16T08:19:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 --2- 172.21.15.32:0/2457564266 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe3000b07f0 0x7fe30015ae70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 --2- 172.21.15.32:0/2457564266 >> v2:172.21.15.186:3300/0 conn(0x7fe30015d690 0x7fe300164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- 172.21.15.32:0/2457564266 >> 172.21.15.32:0/2457564266 conn(0x7fe30006c190 msgr2=0x7fe30006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- 172.21.15.32:0/2457564266 shutdown_connections 2023-10-16T08:19:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- 172.21.15.32:0/2457564266 wait complete. 2023-10-16T08:19:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 Processor -- start 2023-10-16T08:19:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- start start 2023-10-16T08:19:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe300070c40 0x7fe30014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 0x7fe3000ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe30015d690 0x7fe30014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3001f4f30 con 0x7fe3000b07f0 2023-10-16T08:19:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe300150da0 con 0x7fe300070c40 2023-10-16T08:19:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.100+0000 7fe306b72640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000aee60 con 0x7fe30015d690 2023-10-16T08:19:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 0x7fe3000ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 0x7fe3000ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50610/0 (socket says 172.21.15.32:50610) 2023-10-16T08:19:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 -- 172.21.15.32:0/3681370658 learned_addr learned my addr 172.21.15.32:0/3681370658 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe30490e640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe300070c40 0x7fe30014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe30510f640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe30015d690 0x7fe30014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 -- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe30015d690 msgr2=0x7fe30014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe30015d690 0x7fe30014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 -- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe300070c40 msgr2=0x7fe30014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe300070c40 0x7fe30014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 -- 172.21.15.32:0/3681370658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0094d0 con 0x7fe3000b07f0 2023-10-16T08:19:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe30510f640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe30015d690 0x7fe30014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fffff640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 0x7fe3000ae920 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0ac040 tx=0x7fe2ec01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0bd5f0 con 0x7fe3000b07f0 2023-10-16T08:19:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0bd770 con 0x7fe3000b07f0 2023-10-16T08:19:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0bdb20 con 0x7fe3000b07f0 2023-10-16T08:19:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30014e730 con 0x7fe3000b07f0 2023-10-16T08:19:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe300150190 con 0x7fe3000b07f0 2023-10-16T08:19:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe300150f50 con 0x7fe3000b07f0 2023-10-16T08:19:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe2ec0b33f0 con 0x7fe3000b07f0 2023-10-16T08:19:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.104+0000 7fe2fd7fa640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2e40664a0 0x7fe2e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.108+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe2ec11e830 con 0x7fe3000b07f0 2023-10-16T08:19:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.108+0000 7fe30490e640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2e40664a0 0x7fe2e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.108+0000 7fe30490e640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2e40664a0 0x7fe2e4068960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4000f70 tx=0x7fe2f4007b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.112+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe300150f50 con 0x7fe3000b07f0 2023-10-16T08:19:34.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.264+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe300150f50 con 0x7fe2e40664a0 2023-10-16T08:19:34.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe2fd7fa640 1 -- 172.21.15.32:0/3681370658 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50625 (secure 0 0 0) 0x7fe300150f50 con 0x7fe2e40664a0 2023-10-16T08:19:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:34.271 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:19:34.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2e40664a0 msgr2=0x7fe2e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:34.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2e40664a0 0x7fe2e4068960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4000f70 tx=0x7fe2f4007b80 comp rx=0 tx=0).stop 2023-10-16T08:19:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 msgr2=0x7fe3000ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 0x7fe3000ae920 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0ac040 tx=0x7fe2ec01acf0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 shutdown_connections 2023-10-16T08:19:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe2e40664a0 0x7fe2e4068960 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe30015d690 0x7fe30014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3000b07f0 0x7fe3000ae920 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 --2- 172.21.15.32:0/3681370658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe300070c40 0x7fe30014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 >> 172.21.15.32:0/3681370658 conn(0x7fe30006c190 msgr2=0x7fe3000b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 shutdown_connections 2023-10-16T08:19:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:34.268+0000 7fe306b72640 1 -- 172.21.15.32:0/3681370658 wait complete. 2023-10-16T08:19:34.291 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2023-10-16T08:19:34.060747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239372,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735348,"statfs":{"total":773094113280,"available":772848996352,"internally_reserved":0,"allocated":6725632,"data_stored":3641600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.865113"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-16T08:19:25.336518+0000","last_change":"2023-10-16T08:16:12.041120+0000","last_active":"2023-10-16T08:19:25.336518+0000","last_peered":"2023-10-16T08:19:25.336518+0000","last_clean":"2023-10-16T08:19:25.336518+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:19:25.336518+0000","last_undegraded":"2023-10-16T08:19:25.336518+0000","last_fullsized":"2023-10-16T08:19:25.336518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:16:10.859466+0000","last_clean_scrub_stamp":"2023-10-16T08:16:10.859466+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:48.254093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":1286144,"data_stored":897976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83499999999999996}]},{"osd":1,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.76500000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91100000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.746}]},{"osd":3,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85399999999999998,"5min":1.03,"15min":1.03},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":1.056,"15min":1.056},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83899999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88,"5min":1.127,"15min":1.127},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.59599999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.92100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65700000000000003}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68000000000000005,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":1.077,"15min":1.077},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57799999999999996}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.90600000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.91500000000000004,"5min":1.343,"15min":1.343},"last":0.72499999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97899999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.624}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.008,"15min":1.008},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88100000000000001,"5min":1.036,"15min":1.036},"last":0.67500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89100000000000001,"5min":2.774,"15min":2.774},"last":0.91700000000000004}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.58499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94099999999999995,"5min":1.099,"15min":1.099},"last":0.67000000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.504,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92300000000000004,"5min":2.706,"15min":2.706},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.87},"min":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.044,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.89200000000000002},"min":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.71299999999999997}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.997}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.024,"15min":1.024},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.83899999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77800000000000002},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.002,"15min":1.002},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94799999999999995,"5min":1.075,"15min":1.075},"last":0.88700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.443}]},{"osd":2,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.503}]},{"osd":3,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.57299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0089999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.997,"15min":0.997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52700000000000002}]},{"osd":6,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.624},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.80900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44400000000000001,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40600000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77300000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.042,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.745},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.355,"15min":1.355},"last":0.876}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.60599999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.77700000000000002},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.95499999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93500000000000005,"5min":1.391,"15min":1.391},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.60799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0169999999999999,"5min":1.3,"15min":1.3},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.64100000000000001,"15min":0.74299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76500000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.371}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":4.399,"15min":4.399},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.68000000000000005,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72399999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.71799999999999997,"15min":0.81599999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66500000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.499},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.80100000000000005,"15min":0.90300000000000002},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6599999999999999,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.1799999999999999,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.98799999999999999,"15min":1.0700000000000001},"min":{"1min":0.63100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1519999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.96899999999999997,"15min":1.0489999999999999},"min":{"1min":0.53400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.0489999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.624}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":1.0529999999999999},"min":{"1min":0.52400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.1640000000000001,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.97499999999999998,"15min":1.0629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.0649999999999999},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.879,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.749},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98999999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.434,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.46899999999999997,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":1.04,"15min":1.04},"last":0.32100000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.622,"5min":1.073,"15min":1.073},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.374}]},{"osd":4,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":1.556,"15min":1.556},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.627},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83999999999999997,"15min":0.88400000000000001},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.073,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.93000000000000005,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.091,"5min":1.913,"15min":1.913},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.86599999999999999,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92000000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.64700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":897976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76600000000000001,"5min":3.734,"15min":3.734},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.376}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.44400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78900000000000003,"5min":3.492,"15min":3.492},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68000000000000005,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80500000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93500000000000005,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96699999999999997,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:19:34.294 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-16T08:16:10.859466+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=16, tm_sec=10, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=19, tm_sec=28, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:19:34.295 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T08:19:47.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:19:47.472 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:19:54.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:19:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.448+0000 7fe76cc09640 1 Processor -- start 2023-10-16T08:19:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.448+0000 7fe76cc09640 1 -- start start 2023-10-16T08:19:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe76cc09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7680d15d0 0x7fe7680cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe76cc09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7680c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe76cc09640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7680ce770 con 0x7fe76815cb90 2023-10-16T08:19:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe76cc09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680d0600 con 0x7fe7680d15d0 2023-10-16T08:19:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe76cc09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680c6450 con 0x7fe7680d3010 2023-10-16T08:19:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7680d15d0 0x7fe7680cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7680d15d0 0x7fe7680cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57588/0 (socket says 172.21.15.32:57588) 2023-10-16T08:19:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 -- 172.21.15.32:0/153995034 learned_addr learned my addr 172.21.15.32:0/153995034 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe766d9d640 1 --2- 172.21.15.32:0/153995034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7680c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 -- 172.21.15.32:0/153995034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 msgr2=0x7fe7680c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 --2- 172.21.15.32:0/153995034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7680c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 -- 172.21.15.32:0/153995034 >> v1:172.21.15.186:6789/0 conn(0x7fe76815cb90 legacy=0x7fe7680d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 -- 172.21.15.32:0/153995034 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680d08a0 con 0x7fe7680d15d0 2023-10-16T08:19:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe75ffff640 1 --2- 172.21.15.32:0/153995034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680d15d0 0x7fe7680cfb10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe750009870 tx=0x7fe75001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac3cfba8fe937c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.452+0000 7fe765d9b640 1 -- 172.21.15.32:0/153995034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500ab680 con 0x7fe7680d15d0 2023-10-16T08:19:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe765d9b640 1 -- 172.21.15.32:0/153995034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7500ab7e0 con 0x7fe7680d15d0 2023-10-16T08:19:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe765d9b640 1 -- 172.21.15.32:0/153995034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500abb70 con 0x7fe7680d15d0 2023-10-16T08:19:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- 172.21.15.32:0/153995034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680d15d0 msgr2=0x7fe7680cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 --2- 172.21.15.32:0/153995034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680d15d0 0x7fe7680cfb10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe750009870 tx=0x7fe75001c480 comp rx=0 tx=0).stop 2023-10-16T08:19:54.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- 172.21.15.32:0/153995034 shutdown_connections 2023-10-16T08:19:54.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 --2- 172.21.15.32:0/153995034 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7680c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 --2- 172.21.15.32:0/153995034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680d15d0 0x7fe7680cfb10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- 172.21.15.32:0/153995034 >> 172.21.15.32:0/153995034 conn(0x7fe76809bc50 msgr2=0x7fe7680d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:54.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- 172.21.15.32:0/153995034 shutdown_connections 2023-10-16T08:19:54.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- 172.21.15.32:0/153995034 wait complete. 2023-10-16T08:19:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 Processor -- start 2023-10-16T08:19:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- start start 2023-10-16T08:19:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7680d15d0 0x7fe76806b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7681f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 0x7fe7681f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe768157140 con 0x7fe76815cb90 2023-10-16T08:19:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe768157420 con 0x7fe7680d15d0 2023-10-16T08:19:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76cc09640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe768157700 con 0x7fe7680d3010 2023-10-16T08:19:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe766d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 0x7fe7681f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe766d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 0x7fe7681f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57590/0 (socket says 172.21.15.32:57590) 2023-10-16T08:19:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe766d9d640 1 -- 172.21.15.32:0/1655268483 learned_addr learned my addr 172.21.15.32:0/1655268483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe76659c640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7680d15d0 0x7fe76806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.456+0000 7fe75ffff640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7681f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe766d9d640 1 -- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 msgr2=0x7fe7681f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe766d9d640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7681f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe766d9d640 1 -- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7680d15d0 msgr2=0x7fe76806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe766d9d640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7680d15d0 0x7fe76806b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe766d9d640 1 -- 172.21.15.32:0/1655268483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500094d0 con 0x7fe76815cb90 2023-10-16T08:19:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe766d9d640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 0x7fe7681f8c00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe760007e40 tx=0x7fe76000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760014620 con 0x7fe76815cb90 2023-10-16T08:19:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000ee90 con 0x7fe76815cb90 2023-10-16T08:19:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7681579e0 con 0x7fe76815cb90 2023-10-16T08:19:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600149e0 con 0x7fe76815cb90 2023-10-16T08:19:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76806ba80 con 0x7fe76815cb90 2023-10-16T08:19:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe768074010 con 0x7fe76815cb90 2023-10-16T08:19:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe760014b40 con 0x7fe76815cb90 2023-10-16T08:19:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe75effd640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe7480664a0 0x7fe748068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.460+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe760084510 con 0x7fe76815cb90 2023-10-16T08:19:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.464+0000 7fe76659c640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe7480664a0 0x7fe748068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.464+0000 7fe76659c640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe7480664a0 0x7fe748068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe758000fc0 tx=0x7fe758009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.464+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe760051bf0 con 0x7fe76815cb90 2023-10-16T08:19:54.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.620+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 --> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe76815cf70 con 0x7fe7480664a0 2023-10-16T08:19:54.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.620+0000 7fe75effd640 1 -- 172.21.15.32:0/1655268483 <== mgr.4100 v2:172.21.15.186:6832/255423679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50703 (secure 0 0 0) 0x7fe758001d70 con 0x7fe7480664a0 2023-10-16T08:19:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-10-16T08:19:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe7480664a0 msgr2=0x7fe748068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe7480664a0 0x7fe748068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe758000fc0 tx=0x7fe758009040 comp rx=0 tx=0).stop 2023-10-16T08:19:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 msgr2=0x7fe7681f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 0x7fe7681f8c00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe760007e40 tx=0x7fe76000e4b0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 shutdown_connections 2023-10-16T08:19:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe76815cb90 0x7fe7681f8c00 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fe7480664a0 0x7fe748068960 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fe7680d3010 0x7fe7681f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 --2- 172.21.15.32:0/1655268483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7680d15d0 0x7fe76806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 >> 172.21.15.32:0/1655268483 conn(0x7fe76809bc50 msgr2=0x7fe7680d19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 shutdown_connections 2023-10-16T08:19:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.624+0000 7fe76cc09640 1 -- 172.21.15.32:0/1655268483 wait complete. 2023-10-16T08:19:54.646 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-10-16T08:19:54.065455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239360,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735360,"statfs":{"total":773094113280,"available":772849008640,"internally_reserved":0,"allocated":6725632,"data_stored":3641600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002596"},"pg_stats":[{"pgid":"1.0","version":"57'119","reported_seq":209,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-16T08:19:35.260262+0000","last_change":"2023-10-16T08:19:35.260320+0000","last_active":"2023-10-16T08:19:35.260262+0000","last_peered":"2023-10-16T08:19:35.260262+0000","last_clean":"2023-10-16T08:19:35.260262+0000","last_became_active":"2023-10-16T08:16:12.040043+0000","last_became_peered":"2023-10-16T08:16:12.040043+0000","last_unstale":"2023-10-16T08:19:35.260262+0000","last_undegraded":"2023-10-16T08:19:35.260262+0000","last_fullsized":"2023-10-16T08:19:35.260262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'119","last_scrub_stamp":"2023-10-16T08:19:35.260158+0000","last_deep_scrub":"57'119","last_deep_scrub_stamp":"2023-10-16T08:19:35.260158+0000","last_clean_scrub_stamp":"2023-10-16T08:19:35.260158+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:09.006261+0000","scrub_duration":0.0088144090000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1286144,"data_stored":897976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.88800000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65600000000000003,"15min":0.64500000000000002},"min":{"1min":0.47999999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.76500000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91100000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85399999999999998,"5min":1.03,"15min":1.03},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":1.056,"15min":1.056},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83899999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.625}]},{"osd":5,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89900000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83799999999999997,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88,"5min":1.127,"15min":1.127},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.59599999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.81799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83899999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.58499999999999996}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68000000000000005,"15min":0.69299999999999995},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":1.077,"15min":1.077},"last":0.86699999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.48599999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92200000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.503},"max":{"1min":0.90600000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.91500000000000004,"5min":1.343,"15min":1.343},"last":0.92200000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97899999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.94499999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.71299999999999997,"15min":0.72299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.008,"15min":1.008},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88100000000000001,"5min":1.036,"15min":1.036},"last":0.84699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89100000000000001,"5min":2.774,"15min":2.774},"last":0.64100000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.58499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94099999999999995,"5min":1.099,"15min":1.099},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.504,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92300000000000004,"5min":2.706,"15min":2.706},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.87},"min":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.044,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85199999999999998,"15min":0.89200000000000002},"min":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.82999999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.503},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.024,"15min":1.024},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77800000000000002},"min":{"1min":0.52700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.002,"15min":1.002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.78700000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94799999999999995,"5min":1.075,"15min":1.075},"last":0.73499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":1.042},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.51300000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.91400000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.57299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0089999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":0.997,"15min":0.997},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.751},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.46000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83999999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.879}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.44400000000000001,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54200000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41999999999999998}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77300000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.042,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.105},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.355,"15min":1.355},"last":1.121}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.60599999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.038}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.77700000000000002},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":1.0209999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.80200000000000005,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93500000000000005,"5min":1.391,"15min":1.391},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81799999999999995,"15min":0.83399999999999996},"min":{"1min":0.60799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0169999999999999,"5min":1.3,"15min":1.3},"last":0.98599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.64100000000000001,"15min":0.74299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.66500000000000004,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76500000000000001,"5min":4.8550000000000004,"15min":4.8550000000000004},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72299999999999998,"5min":4.399,"15min":4.399},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.68000000000000005,"15min":0.76300000000000001},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72399999999999998,"5min":4.8220000000000001,"15min":4.8220000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.71799999999999997,"15min":0.81599999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66500000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.80100000000000005,"15min":0.90300000000000002},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85899999999999999,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.51900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6599999999999999,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.95699999999999996,"15min":1.0149999999999999},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.1799999999999999,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.83199999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.98799999999999999,"15min":1.0700000000000001},"min":{"1min":0.63100000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1519999999999999,"5min":4.5709999999999997,"15min":4.5709999999999997},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.96899999999999997,"15min":1.0489999999999999},"min":{"1min":0.53400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.0489999999999999,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.84799999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":1.0529999999999999},"min":{"1min":0.52400000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.1640000000000001,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.97499999999999998,"15min":1.0629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1879999999999999,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.0649999999999999},"min":{"1min":0.56000000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.879,"5min":4.7519999999999998,"15min":4.7519999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98999999999999999,"15min":1.0880000000000001},"min":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.86599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":307608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.434,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.46899999999999997,"15min":0.48899999999999999},"min":{"1min":0.32600000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":1.04,"15min":1.04},"last":0.45400000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57299999999999995,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.622,"5min":1.073,"15min":1.073},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59899999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.5}]},{"osd":4,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0269999999999999,"5min":1.556,"15min":1.556},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83999999999999997,"15min":0.88400000000000001},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.073,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.93000000000000005,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.84599999999999997,"15min":0.878},"min":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.754}]},{"osd":7,"last update":"Mon Oct 16 08:19:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85999999999999999,"15min":0.90300000000000002},"min":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.091,"5min":1.913,"15min":1.913},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.86599999999999999,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":0.92000000000000004,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.85899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1286144,"data_stored":897976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76600000000000001,"5min":3.734,"15min":3.734},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.47799999999999998,"15min":0.47999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.44400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78900000000000003,"5min":3.492,"15min":3.492},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68000000000000005,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.86899999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.749}]},{"osd":5,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.501,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93500000000000005,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.81999999999999995}]},{"osd":6,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96699999999999997,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.78800000000000003}]},{"osd":7,"last update":"Mon Oct 16 08:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:19:54.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-16T08:19:54.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82099b8640 1 Processor -- start 2023-10-16T08:19:54.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82099b8640 1 -- start start 2023-10-16T08:19:54.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82099b8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82040b84e0 0x7f82040d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82099b8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040d1670 con 0x7f820415cba0 2023-10-16T08:19:54.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82099b8640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040d1950 con 0x7f82040d0210 2023-10-16T08:19:54.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82099b8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82040d1460 con 0x7f82040b84e0 2023-10-16T08:19:54.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82037fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82040b84e0 0x7f82040d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f8202ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f820415cba0 0x7f82040cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39922/0 (socket says 172.21.15.32:39922) 2023-10-16T08:19:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f8202ffd640 1 -- 172.21.15.32:0/3904817272 learned_addr learned my addr 172.21.15.32:0/3904817272 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82037fe640 1 -- 172.21.15.32:0/3904817272 >> v1:172.21.15.186:6789/0 conn(0x7f82040d0210 legacy=0x7f82040b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:54.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82037fe640 1 -- 172.21.15.32:0/3904817272 >> v1:172.21.15.32:6789/0 conn(0x7f820415cba0 legacy=0x7f82040cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82037fe640 1 -- 172.21.15.32:0/3904817272 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040d1c30 con 0x7f82040b84e0 2023-10-16T08:19:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.788+0000 7f82027fc640 1 -- 172.21.15.32:0/3904817272 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211873 0 0) 0x7f82040d1670 con 0x7f820415cba0 2023-10-16T08:19:54.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82037fe640 1 --2- 172.21.15.32:0/3904817272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82040b84e0 0x7f82040d0f20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009d90 tx=0x7f81fc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f88c0dff877376 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:54.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82027fc640 1 -- 172.21.15.32:0/3904817272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0a5940 con 0x7f82040b84e0 2023-10-16T08:19:54.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82027fc640 1 -- 172.21.15.32:0/3904817272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc0a5aa0 con 0x7f82040b84e0 2023-10-16T08:19:54.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82027fc640 1 -- 172.21.15.32:0/3904817272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0a5e30 con 0x7f82040b84e0 2023-10-16T08:19:54.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 -- 172.21.15.32:0/3904817272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82040b84e0 msgr2=0x7f82040d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 --2- 172.21.15.32:0/3904817272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82040b84e0 0x7f82040d0f20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009d90 tx=0x7f81fc01c880 comp rx=0 tx=0).stop 2023-10-16T08:19:54.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 -- 172.21.15.32:0/3904817272 shutdown_connections 2023-10-16T08:19:54.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 --2- 172.21.15.32:0/3904817272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82040b84e0 0x7f82040d0f20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 -- 172.21.15.32:0/3904817272 >> 172.21.15.32:0/3904817272 conn(0x7f82040719c0 msgr2=0x7f8204071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:54.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 -- 172.21.15.32:0/3904817272 shutdown_connections 2023-10-16T08:19:54.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.792+0000 7f82099b8640 1 -- 172.21.15.32:0/3904817272 wait complete. 2023-10-16T08:19:54.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 Processor -- start 2023-10-16T08:19:54.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- start start 2023-10-16T08:19:54.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82040b84e0 0x7f8204151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82040d0210 0x7f820414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 0x7f820414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040c7e80 con 0x7f820415cba0 2023-10-16T08:19:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820414d1b0 con 0x7f82040d0210 2023-10-16T08:19:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8204151780 con 0x7f82040b84e0 2023-10-16T08:19:54.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 0x7f820414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 0x7f820414a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57602/0 (socket says 172.21.15.32:57602) 2023-10-16T08:19:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 -- 172.21.15.32:0/1836156145 learned_addr learned my addr 172.21.15.32:0/1836156145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:54.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f8202ffd640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82040b84e0 0x7f8204151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f81fbfff640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82040d0210 0x7f820414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 -- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82040b84e0 msgr2=0x7f8204151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82040b84e0 0x7f8204151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 -- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82040d0210 msgr2=0x7f820414a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82040d0210 0x7f820414a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 -- 172.21.15.32:0/1836156145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc009a70 con 0x7f820415cba0 2023-10-16T08:19:54.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82037fe640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 0x7f820414a910 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0b0040 tx=0x7f81fc019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:54.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0ade40 con 0x7f820415cba0 2023-10-16T08:19:54.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc0b7410 con 0x7f820415cba0 2023-10-16T08:19:54.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8204151a60 con 0x7f820415cba0 2023-10-16T08:19:54.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0b7800 con 0x7f820415cba0 2023-10-16T08:19:54.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820414a6b0 con 0x7f820415cba0 2023-10-16T08:19:54.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.796+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8204151f80 con 0x7f820415cba0 2023-10-16T08:19:54.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.800+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81fc0b4050 con 0x7f820415cba0 2023-10-16T08:19:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.800+0000 7f8200ff9640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.800+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f81fc11f110 con 0x7f820415cba0 2023-10-16T08:19:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.800+0000 7f8202ffd640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:54.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.800+0000 7f8202ffd640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000f70 tx=0x7f81f4007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.804+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8204151f80 con 0x7f820415cba0 2023-10-16T08:19:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.960+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f82040c7fc0 con 0x7f820415cba0 2023-10-16T08:19:54.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.964+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f82040c7fc0 con 0x7f820415cba0 2023-10-16T08:19:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f8200ff9640 1 -- 172.21.15.32:0/1836156145 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f81fc0ec820 con 0x7f820415cba0 2023-10-16T08:19:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 msgr2=0x7f81e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000f70 tx=0x7f81f4007df0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 msgr2=0x7f820414a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 0x7f820414a910 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0b0040 tx=0x7f81fc019620 comp rx=0 tx=0).stop 2023-10-16T08:19:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 shutdown_connections 2023-10-16T08:19:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f820415cba0 0x7f820414a910 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82040d0210 0x7f820414a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7f81e40664a0 0x7f81e4068960 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 --2- 172.21.15.32:0/1836156145 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7f82040b84e0 0x7f8204151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 >> 172.21.15.32:0/1836156145 conn(0x7f82040719c0 msgr2=0x7f82040d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 shutdown_connections 2023-10-16T08:19:54.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:54.968+0000 7f82099b8640 1 -- 172.21.15.32:0/1836156145 wait complete. 2023-10-16T08:19:54.990 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T08:19:54.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:19:55.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38f59e640 1 Processor -- start 2023-10-16T08:19:55.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38f59e640 1 -- start start 2023-10-16T08:19:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38f59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 0x7fb3900d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38f59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900ce730 con 0x7fb3900d3170 2023-10-16T08:19:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38f59e640 1 -- --> v1:172.21.15.186:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900d0dd0 con 0x7fb39015c910 2023-10-16T08:19:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38f59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900ce540 con 0x7fb3900cf6f0 2023-10-16T08:19:55.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38dd9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3900d3170 0x7fb3900cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39936/0 (socket says 172.21.15.32:39936) 2023-10-16T08:19:55.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38dd9b640 1 -- 172.21.15.32:0/3753870873 learned_addr learned my addr 172.21.15.32:0/3753870873 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-10-16T08:19:55.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 0x7fb3900d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673350295 0 0) 0x7fb3900ce730 con 0x7fb3900d3170 2023-10-16T08:19:55.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb378003530 con 0x7fb3900d3170 2023-10-16T08:19:55.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.1 v1:172.21.15.186:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995111902 0 0) 0x7fb3900d0dd0 con 0x7fb39015c910 2023-10-16T08:19:55.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 --> v1:172.21.15.186:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3900ce730 con 0x7fb39015c910 2023-10-16T08:19:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4122007396 0 0) 0x7fb378003530 con 0x7fb3900d3170 2023-10-16T08:19:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 msgr2=0x7fb3900d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 0x7fb3900d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 >> v1:172.21.15.186:6789/0 conn(0x7fb39015c910 legacy=0x7fb3900cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900d04a0 con 0x7fb3900d3170 2023-10-16T08:19:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1103963692 0 0) 0x7fb384001680 con 0x7fb3900d3170 2023-10-16T08:19:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 >> v1:172.21.15.32:6789/0 conn(0x7fb3900d3170 legacy=0x7fb3900cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 0x7fb378003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378004180 0x7fb3780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb378005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800bba0 con 0x7fb378004d10 2023-10-16T08:19:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800bd10 con 0x7fb378004180 2023-10-16T08:19:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800bed0 con 0x7fb3900cf6f0 2023-10-16T08:19:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb378005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38dd9b640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378004180 0x7fb3780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38e59c640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 0x7fb378003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 -- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 msgr2=0x7fb378003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900cf6f0 0x7fb378003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 -- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378004180 msgr2=0x7fb3780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:19:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb378004180 0x7fb3780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.132+0000 7fb38ed9d640 1 -- 172.21.15.32:0/3753870873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378003530 con 0x7fb378004d10 2023-10-16T08:19:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb378005190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb388009fc0 tx=0x7fb38801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470e73f06f82e481 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:55.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880a5a80 con 0x7fb378004d10 2023-10-16T08:19:55.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb38800f040 con 0x7fb378004d10 2023-10-16T08:19:55.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38d59a640 1 -- 172.21.15.32:0/3753870873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880a5e40 con 0x7fb378004d10 2023-10-16T08:19:55.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38f59e640 1 -- 172.21.15.32:0/3753870873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 msgr2=0x7fb378005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38f59e640 1 --2- 172.21.15.32:0/3753870873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb378005190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb388009fc0 tx=0x7fb38801ca80 comp rx=0 tx=0).stop 2023-10-16T08:19:55.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38ed9d640 1 -- 172.21.15.32:0/3753870873 reap_dead start 2023-10-16T08:19:55.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38f59e640 1 -- 172.21.15.32:0/3753870873 shutdown_connections 2023-10-16T08:19:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38f59e640 1 -- 172.21.15.32:0/3753870873 >> 172.21.15.32:0/3753870873 conn(0x7fb39009bb50 msgr2=0x7fb3900d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38f59e640 1 -- 172.21.15.32:0/3753870873 shutdown_connections 2023-10-16T08:19:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.136+0000 7fb38f59e640 1 -- 172.21.15.32:0/3753870873 wait complete. 2023-10-16T08:19:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 Processor -- start 2023-10-16T08:19:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 -- start start 2023-10-16T08:19:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb39014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 --2- >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900c8cd0 0x7fb3900c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39015f810 0x7fb39015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900c9670 con 0x7fb378004d10 2023-10-16T08:19:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39014d0f0 con 0x7fb39015f810 2023-10-16T08:19:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 -- --> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901605d0 con 0x7fb3900c8cd0 2023-10-16T08:19:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb39014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb39014cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57618/0 (socket says 172.21.15.32:57618) 2023-10-16T08:19:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 -- 172.21.15.32:0/295570694 learned_addr learned my addr 172.21.15.32:0/295570694 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-10-16T08:19:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38dd9b640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900c8cd0 0x7fb3900c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39015f810 0x7fb39015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 -- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900c8cd0 msgr2=0x7fb3900c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900c8cd0 0x7fb3900c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 -- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39015f810 msgr2=0x7fb39015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39015f810 0x7fb39015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 -- 172.21.15.32:0/295570694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3901608b0 con 0x7fb378004d10 2023-10-16T08:19:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38ed9d640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39015f810 0x7fb39015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:19:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38dd9b640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900c8cd0 0x7fb3900c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:19:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38e59c640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb39014cbb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb3800114c0 tx=0x7fb380011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380006830 con 0x7fb378004d10 2023-10-16T08:19:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb380006990 con 0x7fb378004d10 2023-10-16T08:19:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380006d60 con 0x7fb378004d10 2023-10-16T08:19:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390160b90 con 0x7fb378004d10 2023-10-16T08:19:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.140+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3900b6830 con 0x7fb378004d10 2023-10-16T08:19:55.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.144+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39004e9c0 con 0x7fb378004d10 2023-10-16T08:19:55.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.144+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb380007030 con 0x7fb378004d10 2023-10-16T08:19:55.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.144+0000 7fb37effd640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb3700664d0 0x7fb370068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:19:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.144+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb380086b60 con 0x7fb378004d10 2023-10-16T08:19:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.144+0000 7fb38dd9b640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb3700664d0 0x7fb370068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:19:55.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.144+0000 7fb38dd9b640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb3700664d0 0x7fb370068990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3901f4f70 tx=0x7fb384009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:19:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.148+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb380053fe0 con 0x7fb378004d10 2023-10-16T08:19:55.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.340+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3900cfe20 con 0x7fb378004d10 2023-10-16T08:19:55.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.340+0000 7fb37effd640 1 -- 172.21.15.32:0/295570694 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fb38001a070 con 0x7fb378004d10 2023-10-16T08:19:55.344 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-16T08:19:55.345 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6836/538298583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":538298583},{"type":"v1","addr":"172.21.15.186:6836","nonce":538298583}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1788561351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6835","nonce":1788561351},{"type":"v1","addr":"172.21.15.186:6837","nonce":1788561351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/974500127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":974500127},{"type":"v1","addr":"172.21.15.32:6835","nonce":974500127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6624,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3227135422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":3227135422},{"type":"v1","addr":"172.21.15.32:6834","nonce":3227135422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-10-16T08:19:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 30 2023-10-16T08:19:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb3700664d0 msgr2=0x7fb370068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb3700664d0 0x7fb370068990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3901f4f70 tx=0x7fb384009040 comp rx=0 tx=0).stop 2023-10-16T08:19:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 msgr2=0x7fb39014cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:19:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb39014cbb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb3800114c0 tx=0x7fb380011980 comp rx=0 tx=0).stop 2023-10-16T08:19:55.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 shutdown_connections 2023-10-16T08:19:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb39015f810 0x7fb39015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:6832/255423679,v1:172.21.15.186:6833/255423679] conn(0x7fb3700664d0 0x7fb370068990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.186:3301/0,v1:172.21.15.186:6790/0] conn(0x7fb3900c8cd0 0x7fb3900c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 --2- 172.21.15.32:0/295570694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb378004d10 0x7fb39014cbb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:19:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 >> 172.21.15.32:0/295570694 conn(0x7fb39009bb50 msgr2=0x7fb3900b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:19:55.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 shutdown_connections 2023-10-16T08:19:55.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-10-16T08:19:55.344+0000 7fb38f59e640 1 -- 172.21.15.32:0/295570694 wait complete. 2023-10-16T08:19:55.365 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-16T08:19:55.366 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-16T08:19:55.367 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T08:19:55.367 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.379 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T08:19:55.379 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T08:19:55.380 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.433 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T08:19:55.434 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T08:19:55.435 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.449 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T08:19:55.450 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T08:19:55.450 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.506 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T08:19:55.507 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-16T08:19:55.507 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-16T08:19:55.508 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.566 INFO:tasks.ceph.osd.0:Stopped 2023-10-16T08:19:55.567 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-16T08:19:55.567 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.655 INFO:tasks.ceph.osd.1:Stopped 2023-10-16T08:19:55.656 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-16T08:19:55.656 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:55.737 INFO:tasks.ceph.osd.2:Stopped 2023-10-16T08:19:55.738 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-16T08:19:55.739 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:19:57.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-16T08:20:01.840 INFO:tasks.ceph.osd.3:Stopped 2023-10-16T08:20:01.841 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-16T08:20:01.841 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:01.912 INFO:tasks.ceph.osd.4:Stopped 2023-10-16T08:20:01.913 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-16T08:20:01.913 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:02.006 INFO:tasks.ceph.osd.5:Stopped 2023-10-16T08:20:02.007 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-16T08:20:02.007 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:02.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-16T08:20:02.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-16T08:20:08.108 INFO:tasks.ceph.osd.6:Stopped 2023-10-16T08:20:08.109 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-16T08:20:08.110 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:08.192 INFO:tasks.ceph.osd.7:Stopped 2023-10-16T08:20:08.193 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-16T08:20:08.194 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T08:20:08.194 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:08.249 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T08:20:08.249 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-16T08:20:08.250 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:08.307 INFO:tasks.ceph.mgr.x:Stopped 2023-10-16T08:20:08.307 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-16T08:20:08.308 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-16T08:20:08.308 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:08.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2023-10-16T08:20:08.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-10-16T08:20:08.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-10-16T08:20:08.328 INFO:tasks.ceph.mon.a:Stopped 2023-10-16T08:20:08.329 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-16T08:20:08.329 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:08.346 INFO:tasks.ceph.mon.b:Stopped 2023-10-16T08:20:08.347 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-16T08:20:08.347 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:20:08.404 INFO:tasks.ceph.mon.c:Stopped 2023-10-16T08:20:08.405 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-16T08:20:08.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-16T08:20:08.438 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:20:08.439 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-16T08:20:08.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:20:08.821 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-16T08:20:09.067 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:20:09.068 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-16T08:20:09.275 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:20:09.276 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-16T08:20:09.442 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:20:09.443 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-16T08:20:09.689 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:20:09.690 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-16T08:20:09.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:20:09.874 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-16T08:20:10.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:20:10.056 DEBUG:teuthology.orchestra.run.smithi186:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-16T08:20:10.253 INFO:tasks.ceph:Archiving mon data... 2023-10-16T08:20:10.255 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/data/mon.a.tgz 2023-10-16T08:20:10.255 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-10-16T08:20:10.265 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.uW1y3LObWQ 2023-10-16T08:20:10.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uW1y3LObWQ 2023-10-16T08:20:10.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.uW1y3LObWQ 2023-10-16T08:20:10.485 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.uW1y3LObWQ is 363KB 2023-10-16T08:20:10.550 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.uW1y3LObWQ 2023-10-16T08:20:10.560 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/data/mon.b.tgz 2023-10-16T08:20:10.560 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-10-16T08:20:10.569 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.pmot6UAVPF 2023-10-16T08:20:10.570 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.pmot6UAVPF 2023-10-16T08:20:10.727 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.pmot6UAVPF 2023-10-16T08:20:10.808 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.pmot6UAVPF is 478KB 2023-10-16T08:20:10.873 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.pmot6UAVPF 2023-10-16T08:20:10.882 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/data/mon.c.tgz 2023-10-16T08:20:10.883 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2023-10-16T08:20:10.935 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.Xy9DUcwh0t 2023-10-16T08:20:10.935 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Xy9DUcwh0t 2023-10-16T08:20:11.096 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.Xy9DUcwh0t 2023-10-16T08:20:11.176 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.Xy9DUcwh0t is 478KB 2023-10-16T08:20:11.245 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.Xy9DUcwh0t 2023-10-16T08:20:11.254 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-16T08:20:11.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T08:20:11.259 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T08:20:11.314 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-16T08:20:11.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T08:20:11.327 DEBUG:teuthology.orchestra.run.smithi186:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T08:20:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T08:20:11.389 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T08:20:11.396 INFO:tasks.ceph:Archiving crash dumps... 2023-10-16T08:20:11.398 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/remote/smithi032/crash 2023-10-16T08:20:11.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T08:20:11.439 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/remote/smithi186/crash 2023-10-16T08:20:11.440 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T08:20:11.460 INFO:tasks.ceph:Shutting down logrotate 2023-10-16T08:20:11.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:20:11.487 DEBUG:teuthology.orchestra.run.smithi186:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:20:11.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T08:20:11.598 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T08:20:11.618 INFO:tasks.ceph:Compressing logs... 2023-10-16T08:20:11.618 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T08:20:11.643 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T08:20:11.664 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52284.log 2023-10-16T08:20:11.665 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2023-10-16T08:20:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53004.log 2023-10-16T08:20:11.666 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52284.log.gz 2023-10-16T08:20:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2023-10-16T08:20:11.667 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2023-10-16T08:20:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2023-10-16T08:20:11.668 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53004.log.gz 2023-10-16T08:20:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49963.log 2023-10-16T08:20:11.670 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2023-10-16T08:20:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44996.log 2023-10-16T08:20:11.671 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2023-10-16T08:20:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51645.log 2023-10-16T08:20:11.672 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49963.log.gz 2023-10-16T08:20:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42705.log 2023-10-16T08:20:11.674 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44996.log.gz 2023-10-16T08:20:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2023-10-16T08:20:11.674 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51645.log.gz 2023-10-16T08:20:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42705.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42705.log.gzgzip 2023-10-16T08:20:11.676 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51814.log 2023-10-16T08:20:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2023-10-16T08:20:11.676 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2023-10-16T08:20:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43582.log 2023-10-16T08:20:11.678 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51814.log.gz 2023-10-16T08:20:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47002.log 2023-10-16T08:20:11.678 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2023-10-16T08:20:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43582.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43582.log.gz -5 2023-10-16T08:20:11.680 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49049.log 2023-10-16T08:20:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52239.log 2023-10-16T08:20:11.680 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47002.log.gz 2023-10-16T08:20:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49049.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-16T08:20:11.681 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49049.log.gz 2023-10-16T08:20:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52239.log.gzgzip 2023-10-16T08:20:11.682 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2023-10-16T08:20:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47385.log 2023-10-16T08:20:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33137.log 2023-10-16T08:20:11.684 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz 2023-10-16T08:20:11.685 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2023-10-16T08:20:11.685 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47385.log.gz 2023-10-16T08:20:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.33137.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33137.log.gz 2023-10-16T08:20:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50805.log 2023-10-16T08:20:11.687 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52978.log: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-16T08:20:11.687 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gzgzip 2023-10-16T08:20:11.687 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42207.log 2023-10-16T08:20:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46898.log 2023-10-16T08:20:11.689 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50805.log.gz 2023-10-16T08:20:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47236.log 2023-10-16T08:20:11.689 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42207.log.gz 2023-10-16T08:20:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50701.log 2023-10-16T08:20:11.691 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46898.log.gz 2023-10-16T08:20:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50060.log 2023-10-16T08:20:11.691 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47236.log.gz 2023-10-16T08:20:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50701.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50701.log.gz -5 2023-10-16T08:20:11.692 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48330.log 2023-10-16T08:20:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52640.log 2023-10-16T08:20:11.693 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50060.log.gz 2023-10-16T08:20:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45341.log 2023-10-16T08:20:11.695 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48330.log.gz 2023-10-16T08:20:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2023-10-16T08:20:11.696 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52640.log.gz 2023-10-16T08:20:11.697 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51514.log 2023-10-16T08:20:11.697 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45341.log.gz 2023-10-16T08:20:11.697 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51334.log 2023-10-16T08:20:11.698 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2023-10-16T08:20:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47288.log 2023-10-16T08:20:11.699 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2023-10-16T08:20:11.700 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51514.log.gz 2023-10-16T08:20:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.51334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45471.log 2023-10-16T08:20:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47288.log: 90.0% -- replaced with /var/log/ceph/ceph-client.0.51334.log.gz 2023-10-16T08:20:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2023-10-16T08:20:11.701 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47288.log.gz 2023-10-16T08:20:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50086.log 2023-10-16T08:20:11.702 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2023-10-16T08:20:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose 89.7% -- /var/log/ceph/ceph-client.admin.43383.log -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2023-10-16T08:20:11.706 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52005.log 2023-10-16T08:20:11.706 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50086.log.gz 2023-10-16T08:20:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43383.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz /var/log/ceph/ceph-client.admin.51065.log 2023-10-16T08:20:11.707 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52005.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52005.log.gz 2023-10-16T08:20:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43592.log 2023-10-16T08:20:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45627.log 2023-10-16T08:20:11.708 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51065.log.gz 2023-10-16T08:20:11.710 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-16T08:20:11.710 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38277.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-16T08:20:11.710 INFO:teuthology.orchestra.run.smithi186.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2023-10-16T08:20:11.711 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38253.log 2023-10-16T08:20:11.711 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2023-10-16T08:20:11.711 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38253.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-16T08:20:11.712 INFO:teuthology.orchestra.run.smithi186.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38253.log.gz 2023-10-16T08:20:11.712 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38205.log 2023-10-16T08:20:11.712 INFO:teuthology.orchestra.run.smithi186.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2023-10-16T08:20:11.713 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44176.log 2023-10-16T08:20:11.713 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38205.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38205.log.gz 2023-10-16T08:20:11.713 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-16T08:20:11.713 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.3.44176.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T08:20:11.714 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mgr.x.log: 91.5% -- replaced with /var/log/ceph/ceph-client.3.44176.log.gz 2023-10-16T08:20:11.714 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2023-10-16T08:20:11.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-16T08:20:11.718 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38301.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T08:20:11.718 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44062.log 2023-10-16T08:20:11.719 INFO:teuthology.orchestra.run.smithi186.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2023-10-16T08:20:11.719 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T08:20:11.720 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.1.44062.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-16T08:20:11.720 INFO:teuthology.orchestra.run.smithi186.stderr: 90.5%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.1.44062.log.gz 2023-10-16T08:20:11.721 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2023-10-16T08:20:11.727 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.38181.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2023-10-16T08:20:11.727 INFO:teuthology.orchestra.run.smithi186.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T08:20:11.728 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-16T08:20:11.728 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2023-10-16T08:20:11.728 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-16T08:20:11.728 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38229.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-16T08:20:11.729 INFO:teuthology.orchestra.run.smithi186.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2023-10-16T08:20:11.729 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38133.log 2023-10-16T08:20:11.729 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-16T08:20:11.731 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.admin.38133.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38133.log.gz 2023-10-16T08:20:11.731 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-16T08:20:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2023-10-16T08:20:11.735 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43592.log.gz 2023-10-16T08:20:11.735 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45975.log 2023-10-16T08:20:11.735 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45627.log.gz 2023-10-16T08:20:11.736 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45289.log 2023-10-16T08:20:11.736 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gz 2023-10-16T08:20:11.736 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45975.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-16T08:20:11.736 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45975.log.gz 2023-10-16T08:20:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45315.log 2023-10-16T08:20:11.737 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45289.log.gz 2023-10-16T08:20:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2023-10-16T08:20:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37946.log 2023-10-16T08:20:11.738 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45315.log.gz 2023-10-16T08:20:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48944.log 2023-10-16T08:20:11.739 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2023-10-16T08:20:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2023-10-16T08:20:11.739 INFO:teuthology.orchestra.run.smithi032.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.37946.log.gz 2023-10-16T08:20:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48944.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2023-10-16T08:20:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50164.log 2023-10-16T08:20:11.741 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46555.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz /var/log/ceph/ceph-client.admin.47665.log 2023-10-16T08:20:11.741 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50164.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.50164.log.gz -- 2023-10-16T08:20:11.742 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.49075.log 2023-10-16T08:20:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2023-10-16T08:20:11.743 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51979.log 2023-10-16T08:20:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46084.log 89.5% 2023-10-16T08:20:11.743 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49075.log.gz 2023-10-16T08:20:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46633.log 2023-10-16T08:20:11.744 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51979.log.gz 2023-10-16T08:20:11.744 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46084.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46084.log.gzgzip 2023-10-16T08:20:11.744 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52588.log 2023-10-16T08:20:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46633.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.46633.log.gz /var/log/ceph/ceph-client.admin.48413.log 2023-10-16T08:20:11.745 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52510.log 2023-10-16T08:20:11.746 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52588.log.gz 2023-10-16T08:20:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52614.log 2023-10-16T08:20:11.746 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48413.log.gz 2023-10-16T08:20:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52510.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52510.log.gz 2023-10-16T08:20:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50986.log 2023-10-16T08:20:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52614.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52614.log.gz 2023-10-16T08:20:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45896.log 2023-10-16T08:20:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50986.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.50986.log.gz -- 2023-10-16T08:20:11.748 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.52562.log 2023-10-16T08:20:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45896.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45896.log.gz 2023-10-16T08:20:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42575.log 2023-10-16T08:20:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46950.log 2023-10-16T08:20:11.750 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52562.log.gz 2023-10-16T08:20:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46318.log 2023-10-16T08:20:11.751 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42575.log.gz 2023-10-16T08:20:11.751 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43411.log 2023-10-16T08:20:11.751 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46950.log.gz 2023-10-16T08:20:11.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46318.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46318.log.gz 2023-10-16T08:20:11.752 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38171.log 2023-10-16T08:20:11.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2023-10-16T08:20:11.752 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43411.log.gz 2023-10-16T08:20:11.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44955.log 88.0% 2023-10-16T08:20:11.753 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38171.log.gz 2023-10-16T08:20:11.760 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49703.log 2023-10-16T08:20:11.761 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2023-10-16T08:20:11.761 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.44955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2023-10-16T08:20:11.762 INFO:teuthology.orchestra.run.smithi032.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.2.44955.log.gz 2023-10-16T08:20:11.762 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42431.log 2023-10-16T08:20:11.762 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49703.log.gz 2023-10-16T08:20:11.762 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2023-10-16T08:20:11.763 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2023-10-16T08:20:11.763 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42431.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42431.log.gz 2023-10-16T08:20:11.763 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43489.log 2023-10-16T08:20:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47586.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2023-10-16T08:20:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48649.log 2023-10-16T08:20:11.764 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49023.log 2023-10-16T08:20:11.765 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43489.log.gz 2023-10-16T08:20:11.765 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43248.log 2023-10-16T08:20:11.765 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48649.log.gz 2023-10-16T08:20:11.765 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44555.log 2023-10-16T08:20:11.766 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49023.log.gz 2023-10-16T08:20:11.766 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43248.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43248.log.gz 2023-10-16T08:20:11.766 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52213.log 2023-10-16T08:20:11.767 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50242.log 2023-10-16T08:20:11.767 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44555.log.gz 2023-10-16T08:20:11.767 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45367.log 2023-10-16T08:20:11.767 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52213.log.gz 2023-10-16T08:20:11.768 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49755.log 2023-10-16T08:20:11.768 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50242.log.gz 2023-10-16T08:20:11.768 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45367.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2023-10-16T08:20:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52822.log 2023-10-16T08:20:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44373.log 2023-10-16T08:20:11.769 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49755.log.gz 2023-10-16T08:20:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52822.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52822.log.gz 2023-10-16T08:20:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42601.log 2023-10-16T08:20:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2023-10-16T08:20:11.770 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44373.log.gz 2023-10-16T08:20:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43996.log 2023-10-16T08:20:11.771 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42601.log.gz 2023-10-16T08:20:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48466.log 2023-10-16T08:20:11.772 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2023-10-16T08:20:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42493.log 2023-10-16T08:20:11.772 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43996.log.gz 2023-10-16T08:20:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48466.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48466.log.gz 2023-10-16T08:20:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51013.log 2023-10-16T08:20:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 2023-10-16T08:20:11.773 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42493.log.gz 2023-10-16T08:20:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2023-10-16T08:20:11.774 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51013.log.gz 2023-10-16T08:20:11.780 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45133.log: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2023-10-16T08:20:11.780 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44295.log 2023-10-16T08:20:11.780 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42987.log 2023-10-16T08:20:11.781 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2023-10-16T08:20:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2023-10-16T08:20:11.781 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44295.log.gz 2023-10-16T08:20:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46528.log 2023-10-16T08:20:11.782 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42987.log.gz 2023-10-16T08:20:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43588.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2023-10-16T08:20:11.782 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2023-10-16T08:20:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46528.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-16T08:20:11.783 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46528.log.gz 2023-10-16T08:20:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49320.log 2023-10-16T08:20:11.783 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2023-10-16T08:20:11.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45211.log 2023-10-16T08:20:11.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49320.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49320.log.gz 2023-10-16T08:20:11.784 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2023-10-16T08:20:11.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45211.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.48758.log 2023-10-16T08:20:11.785 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45211.log.gz 2023-10-16T08:20:11.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44217.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2023-10-16T08:20:11.785 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44347.log 2023-10-16T08:20:11.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48758.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48758.log.gz 2023-10-16T08:20:11.786 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49885.log 2023-10-16T08:20:11.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44347.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44347.log.gz 2023-10-16T08:20:11.787 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47939.log 2023-10-16T08:20:11.787 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46502.log 2023-10-16T08:20:11.787 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49885.log.gz 2023-10-16T08:20:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47939.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47939.log.gz 2023-10-16T08:20:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52109.log 2023-10-16T08:20:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46502.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46502.log.gz 2023-10-16T08:20:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42627.log 2023-10-16T08:20:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52109.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52109.log.gz 2023-10-16T08:20:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47210.log 2023-10-16T08:20:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42627.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42627.log.gz -5 2023-10-16T08:20:11.790 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2023-10-16T08:20:11.790 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-osd.4.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-16T08:20:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47210.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47210.log.gz 2023-10-16T08:20:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2023-10-16T08:20:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2023-10-16T08:20:11.801 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2023-10-16T08:20:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46053.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2023-10-16T08:20:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51462.log 2023-10-16T08:20:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48570.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2023-10-16T08:20:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43787.log 2023-10-16T08:20:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51462.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51462.log.gz 2023-10-16T08:20:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51953.log 2023-10-16T08:20:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43787.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43787.log.gz 2023-10-16T08:20:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51619.log 2023-10-16T08:20:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51953.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51953.log.gz 2023-10-16T08:20:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2023-10-16T08:20:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42179.log 2023-10-16T08:20:11.805 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51619.log.gz 2023-10-16T08:20:11.805 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50621.log 2023-10-16T08:20:11.806 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2023-10-16T08:20:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42179.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42179.log.gz 2023-10-16T08:20:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-16T08:20:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46716.log 2023-10-16T08:20:11.807 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50621.log.gz 2023-10-16T08:20:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2023-10-16T08:20:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46716.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46716.log.gz 2023-10-16T08:20:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52057.log 2023-10-16T08:20:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42679.log 2023-10-16T08:20:11.808 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2023-10-16T08:20:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52057.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52057.log.gz 2023-10-16T08:20:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46292.log 2023-10-16T08:20:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42679.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42679.log.gz 2023-10-16T08:20:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51170.log 2023-10-16T08:20:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52187.log 2023-10-16T08:20:11.814 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46292.log.gz 2023-10-16T08:20:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51170.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51170.log.gz 2023-10-16T08:20:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47028.log 2023-10-16T08:20:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46027.log 2023-10-16T08:20:11.815 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52187.log.gz 2023-10-16T08:20:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47028.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47028.log.gz 2023-10-16T08:20:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48813.log 2023-10-16T08:20:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44814.log 2023-10-16T08:20:11.816 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46027.log.gz 2023-10-16T08:20:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48356.log 2023-10-16T08:20:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.44814.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.48813.log.gz 2023-10-16T08:20:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51488.log 2023-10-16T08:20:11.818 INFO:teuthology.orchestra.run.smithi032.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.44814.log.gz 2023-10-16T08:20:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2023-10-16T08:20:11.818 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48356.log.gz 2023-10-16T08:20:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51488.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51488.log.gz 2023-10-16T08:20:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49609.log 2023-10-16T08:20:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44321.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2023-10-16T08:20:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47106.log 2023-10-16T08:20:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52954.log 2023-10-16T08:20:11.820 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49609.log.gz 2023-10-16T08:20:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47106.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47106.log.gz 2023-10-16T08:20:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51677.log 2023-10-16T08:20:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52954.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52954.log.gz 2023-10-16T08:20:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47967.log 2023-10-16T08:20:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51677.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51677.log.gz 2023-10-16T08:20:11.824 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48623.log 2023-10-16T08:20:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49833.log 2023-10-16T08:20:11.825 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47967.log.gz 2023-10-16T08:20:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48623.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48623.log.gz 2023-10-16T08:20:11.825 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2023-10-16T08:20:11.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49833.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49833.log.gz 2023-10-16T08:20:11.826 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49583.log 2023-10-16T08:20:11.826 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47158.log 2023-10-16T08:20:11.827 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2023-10-16T08:20:11.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49583.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49583.log.gz 2023-10-16T08:20:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52083.log 2023-10-16T08:20:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47158.log.gz 2023-10-16T08:20:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2023-10-16T08:20:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51866.log 2023-10-16T08:20:11.830 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52083.log.gz 2023-10-16T08:20:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49781.log 2023-10-16T08:20:11.830 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2023-10-16T08:20:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51866.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51866.log.gz 2023-10-16T08:20:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43515.log 2023-10-16T08:20:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50453.log 2023-10-16T08:20:11.833 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49781.log.gz 2023-10-16T08:20:11.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2023-10-16T08:20:11.834 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43515.log.gz 2023-10-16T08:20:11.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2023-10-16T08:20:11.835 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2023-10-16T08:20:11.836 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51248.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2023-10-16T08:20:11.836 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48866.log 2023-10-16T08:20:11.837 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2023-10-16T08:20:11.837 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46344.log: /var/log/ceph/ceph-client.admin.48866.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2023-10-16T08:20:11.838 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48866.log.gz 2023-10-16T08:20:11.838 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46398.log 2023-10-16T08:20:11.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42653.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2023-10-16T08:20:11.840 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52744.log 2023-10-16T08:20:11.841 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46398.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46398.log.gz 2023-10-16T08:20:11.842 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48045.log 2023-10-16T08:20:11.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52744.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52744.log.gz 2023-10-16T08:20:11.844 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43065.log 2023-10-16T08:20:11.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46820.log 2023-10-16T08:20:11.846 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48045.log.gz 2023-10-16T08:20:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43065.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43065.log.gz 2023-10-16T08:20:11.847 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44399.log 2023-10-16T08:20:11.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46820.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46820.log.gz 2023-10-16T08:20:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49557.log 2023-10-16T08:20:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44399.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.44399.log.gz 2023-10-16T08:20:11.849 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50649.log 2023-10-16T08:20:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2023-10-16T08:20:11.850 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49557.log.gz 2023-10-16T08:20:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50649.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50649.log.gz 2023-10-16T08:20:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43196.log 2023-10-16T08:20:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46240.log 2023-10-16T08:20:11.853 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2023-10-16T08:20:11.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43196.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43196.log.gz 2023-10-16T08:20:11.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50753.log 2023-10-16T08:20:11.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46240.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46240.log.gz 2023-10-16T08:20:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-16T08:20:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50753.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50753.log.gz 2023-10-16T08:20:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52432.log 2023-10-16T08:20:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2023-10-16T08:20:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52432.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52432.log.gz 2023-10-16T08:20:11.860 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43091.log 2023-10-16T08:20:11.860 INFO:teuthology.orchestra.run.smithi186.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-16T08:20:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45923.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gz 2023-10-16T08:20:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43118.log 2023-10-16T08:20:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43091.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43091.log.gz 2023-10-16T08:20:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44694.log 2023-10-16T08:20:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43118.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43118.log.gz 2023-10-16T08:20:11.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52770.log 2023-10-16T08:20:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48918.log 2023-10-16T08:20:11.866 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44694.log.gz 2023-10-16T08:20:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52770.log.gz 2023-10-16T08:20:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47853.log 2023-10-16T08:20:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51541.log 2023-10-16T08:20:11.869 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48918.log.gz 2023-10-16T08:20:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47853.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47853.log.gz 2023-10-16T08:20:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49186.log 2023-10-16T08:20:11.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45263.log 2023-10-16T08:20:11.872 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51541.log.gz 2023-10-16T08:20:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49186.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49186.log.gz 2023-10-16T08:20:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51196.log 2023-10-16T08:20:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46162.log 2023-10-16T08:20:11.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45263.log.gz 2023-10-16T08:20:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51196.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51196.log.gz 2023-10-16T08:20:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2023-10-16T08:20:11.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46162.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46162.log.gz 2023-10-16T08:20:11.877 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49477.log 2023-10-16T08:20:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2023-10-16T08:20:11.879 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2023-10-16T08:20:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49477.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49477.log.gz 2023-10-16T08:20:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51144.log 2023-10-16T08:20:11.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50479.log 2023-10-16T08:20:11.882 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2023-10-16T08:20:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51144.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51144.log.gz 2023-10-16T08:20:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43170.log 2023-10-16T08:20:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50479.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50479.log.gz 2023-10-16T08:20:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48544.log 2023-10-16T08:20:11.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2023-10-16T08:20:11.886 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43170.log.gz 2023-10-16T08:20:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48544.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2023-10-16T08:20:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2023-10-16T08:20:11.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42089.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 2023-10-16T08:20:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2023-10-16T08:20:11.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45523.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2023-10-16T08:20:11.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46214.log 2023-10-16T08:20:11.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48199.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2023-10-16T08:20:11.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48149.log 2023-10-16T08:20:11.893 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2023-10-16T08:20:11.893 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46214.log.gz 2023-10-16T08:20:11.894 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52404.log 2023-10-16T08:20:11.895 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48149.log.gz 2023-10-16T08:20:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45792.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2023-10-16T08:20:11.896 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2023-10-16T08:20:11.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46424.log 2023-10-16T08:20:11.897 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52404.log.gz 2023-10-16T08:20:11.899 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45393.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2023-10-16T08:20:11.899 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50881.log 2023-10-16T08:20:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46424.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46424.log.gz 2023-10-16T08:20:11.900 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45159.log 2023-10-16T08:20:11.902 INFO:teuthology.orchestra.run.smithi186.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-16T08:20:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50881.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50881.log.gz 2023-10-16T08:20:11.903 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48255.log 2023-10-16T08:20:11.904 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45159.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45159.log.gz 2023-10-16T08:20:11.904 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-16T08:20:11.905 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47411.log 2023-10-16T08:20:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2023-10-16T08:20:11.907 INFO:teuthology.orchestra.run.smithi032.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.48255.log.gz 2023-10-16T08:20:11.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz 2023-10-16T08:20:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47054.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2023-10-16T08:20:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46136.log 2023-10-16T08:20:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48440.log 2023-10-16T08:20:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46136.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46136.log.gz 2023-10-16T08:20:11.914 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51222.log 2023-10-16T08:20:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48440.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48440.log.gz 2023-10-16T08:20:11.915 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50507.log 2023-10-16T08:20:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2023-10-16T08:20:11.917 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51222.log.gz 2023-10-16T08:20:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50507.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50507.log.gz 2023-10-16T08:20:11.919 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49859.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2023-10-16T08:20:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52666.log 2023-10-16T08:20:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2023-10-16T08:20:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2023-10-16T08:20:11.923 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52666.log.gz 2023-10-16T08:20:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43576.log 2023-10-16T08:20:11.925 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2023-10-16T08:20:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45870.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gzgzip 2023-10-16T08:20:11.926 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45497.log 2023-10-16T08:20:11.927 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2023-10-16T08:20:11.928 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43576.log.gz 2023-10-16T08:20:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51279.log 2023-10-16T08:20:11.929 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45497.log.gz 2023-10-16T08:20:11.930 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48492.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2023-10-16T08:20:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42377.log 2023-10-16T08:20:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2023-10-16T08:20:11.932 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51279.log.gz 2023-10-16T08:20:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42377.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42377.log.gz 2023-10-16T08:20:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51782.log 2023-10-16T08:20:11.935 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44243.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2023-10-16T08:20:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42907.log 2023-10-16T08:20:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50034.log 2023-10-16T08:20:11.937 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51782.log.gz 2023-10-16T08:20:11.940 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42907.log: /var/log/ceph/ceph-client.admin.50034.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50034.log.gz 2023-10-16T08:20:11.940 INFO:teuthology.orchestra.run.smithi032.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.42907.log.gz 2023-10-16T08:20:11.940 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52900.log 2023-10-16T08:20:11.941 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44009.log 2023-10-16T08:20:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45949.log 2023-10-16T08:20:11.943 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52900.log.gz 2023-10-16T08:20:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50675.log 2023-10-16T08:20:11.945 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44009.log.gz 2023-10-16T08:20:11.946 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46110.log 2023-10-16T08:20:11.947 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45949.log.gz 2023-10-16T08:20:11.947 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46476.log 89.4% 2023-10-16T08:20:11.948 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50675.log.gz 2023-10-16T08:20:11.949 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49346.log 2023-10-16T08:20:11.949 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46110.log.gz 2023-10-16T08:20:11.950 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46476.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46476.log.gz 2023-10-16T08:20:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49346.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49346.log.gz 2023-10-16T08:20:11.952 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48892.log 2023-10-16T08:20:11.953 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47080.log 2023-10-16T08:20:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48019.log 2023-10-16T08:20:11.955 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48892.log.gz 2023-10-16T08:20:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47080.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47080.log.gz 2023-10-16T08:20:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48019.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48019.log.gz 2023-10-16T08:20:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43578.log 2023-10-16T08:20:11.960 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2023-10-16T08:20:11.962 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43595.log 2023-10-16T08:20:11.962 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43578.log.gz 2023-10-16T08:20:11.963 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49640.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2023-10-16T08:20:11.964 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44425.log 2023-10-16T08:20:11.965 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43595.log.gz 2023-10-16T08:20:11.966 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42961.log 2023-10-16T08:20:11.967 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50371.log 2023-10-16T08:20:11.968 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44425.log.gz 2023-10-16T08:20:11.969 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52161.log 2023-10-16T08:20:11.969 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42961.log.gz 2023-10-16T08:20:11.970 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50371.log.gz 2023-10-16T08:20:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52161.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52161.log.gz 2023-10-16T08:20:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T08:20:11.973 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42781.log 2023-10-16T08:20:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49729.log 2023-10-16T08:20:11.976 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42781.log: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T08:20:11.977 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2023-10-16T08:20:11.977 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42781.log.gz 2023-10-16T08:20:11.978 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49729.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49729.log.gz 2023-10-16T08:20:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42549.log 2023-10-16T08:20:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52336.log 2023-10-16T08:20:11.980 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2023-10-16T08:20:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42549.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42549.log.gz 2023-10-16T08:20:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52848.log 2023-10-16T08:20:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50307.log 2023-10-16T08:20:11.983 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52336.log.gz 2023-10-16T08:20:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52848.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52718.log 2023-10-16T08:20:11.985 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52848.log.gz 2023-10-16T08:20:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51383.log 2023-10-16T08:20:11.986 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50307.log.gz 2023-10-16T08:20:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49127.log 2023-10-16T08:20:11.988 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52718.log.gz 2023-10-16T08:20:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49101.log 2023-10-16T08:20:11.990 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51383.log.gz 2023-10-16T08:20:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2023-10-16T08:20:11.991 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49127.log.gz 2023-10-16T08:20:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47314.log 2023-10-16T08:20:11.992 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49101.log.gz 2023-10-16T08:20:11.993 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44451.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2023-10-16T08:20:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50779.log 2023-10-16T08:20:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47314.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47314.log.gz -5 2023-10-16T08:20:11.995 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45844.log 2023-10-16T08:20:11.996 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50779.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50779.log.gz 2023-10-16T08:20:11.997 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51567.log 2023-10-16T08:20:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52926.log 2023-10-16T08:20:11.999 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45844.log.gz 2023-10-16T08:20:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51567.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51567.log.gz 2023-10-16T08:20:12.001 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52926.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52926.log.gz 2023-10-16T08:20:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2023-10-16T08:20:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 2023-10-16T08:20:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51436.log 2023-10-16T08:20:12.006 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2023-10-16T08:20:12.007 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51039.log 2023-10-16T08:20:12.007 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2023-10-16T08:20:12.008 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50831.log 2023-10-16T08:20:12.008 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51436.log.gz 2023-10-16T08:20:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51039.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51039.log.gz -5 2023-10-16T08:20:12.010 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2023-10-16T08:20:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43013.log 2023-10-16T08:20:12.012 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50831.log.gz 2023-10-16T08:20:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46685.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2023-10-16T08:20:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43013.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43013.log.gz 2023-10-16T08:20:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46266.log 2023-10-16T08:20:12.016 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51091.log 2023-10-16T08:20:12.018 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43803.log 2023-10-16T08:20:12.018 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46266.log.gz 2023-10-16T08:20:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48675.log 2023-10-16T08:20:12.020 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51091.log.gz 2023-10-16T08:20:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43803.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43803.log.gz 2023-10-16T08:20:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2023-10-16T08:20:12.023 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43039.log 2023-10-16T08:20:12.023 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48675.log.gz 2023-10-16T08:20:12.024 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50535.log 2023-10-16T08:20:12.025 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2023-10-16T08:20:12.026 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2023-10-16T08:20:12.026 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43039.log.gz 2023-10-16T08:20:12.027 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50535.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50535.log.gz 2023-10-16T08:20:12.027 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48382.log 2023-10-16T08:20:12.029 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2023-10-16T08:20:12.029 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2023-10-16T08:20:12.030 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2023-10-16T08:20:12.031 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48382.log.gz 2023-10-16T08:20:12.031 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45549.log 2023-10-16T08:20:12.032 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2023-10-16T08:20:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43222.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2023-10-16T08:20:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45549.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45549.log.gz 2023-10-16T08:20:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50138.log 2023-10-16T08:20:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44503.log 2023-10-16T08:20:12.038 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46742.log 2023-10-16T08:20:12.038 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50138.log.gz 2023-10-16T08:20:12.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44503.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44503.log.gz 2023-10-16T08:20:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46742.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46742.log.gz 2023-10-16T08:20:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2023-10-16T08:20:12.043 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2023-10-16T08:20:12.045 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47437.log 2023-10-16T08:20:12.045 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2023-10-16T08:20:12.046 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47534.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2023-10-16T08:20:12.047 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47437.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2023-10-16T08:20:12.048 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2023-10-16T08:20:12.049 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50936.log 2023-10-16T08:20:12.051 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49911.log 2023-10-16T08:20:12.051 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2023-10-16T08:20:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.50936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2023-10-16T08:20:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49911.log: 88.2% 91.7% -- replaced with /var/log/ceph/ceph-client.0.50936.log.gz 2023-10-16T08:20:12.054 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49911.log.gz 2023-10-16T08:20:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49989.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2023-10-16T08:20:12.056 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T08:20:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49503.log 2023-10-16T08:20:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52536.log 2023-10-16T08:20:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49503.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49503.log.gz 2023-10-16T08:20:12.061 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T08:20:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52536.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52536.log.gz 2023-10-16T08:20:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-16T08:20:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46450.log 2023-10-16T08:20:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50595.log 2023-10-16T08:20:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46450.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46450.log.gz 2023-10-16T08:20:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50595.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50595.log.gz 2023-10-16T08:20:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50216.log 2023-10-16T08:20:12.070 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52362.log 2023-10-16T08:20:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50216.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50216.log.gz 2023-10-16T08:20:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-16T08:20:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52362.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52362.log.gz 2023-10-16T08:20:12.074 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47825.log 2023-10-16T08:20:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.47825.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47825.log.gz 2023-10-16T08:20:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50569.log 2023-10-16T08:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46370.log 2023-10-16T08:20:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50569.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50569.log.gz 2023-10-16T08:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46370.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46370.log.gz 2023-10-16T08:20:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43989.log 2023-10-16T08:20:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2023-10-16T08:20:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44477.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2023-10-16T08:20:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43989.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43989.log.gz 2023-10-16T08:20:12.091 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2023-10-16T08:20:12.093 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49677.log 2023-10-16T08:20:12.094 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2023-10-16T08:20:12.101 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48097.log 2023-10-16T08:20:12.101 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48097.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48097.log.gz 2023-10-16T08:20:12.101 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47132.log 2023-10-16T08:20:12.105 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2%/var/log/ceph/ceph-client.admin.47132.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47132.log.gz 2023-10-16T08:20:12.106 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49677.log.gz 2023-10-16T08:20:12.106 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43144.log 2023-10-16T08:20:12.109 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43144.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43144.log.gz 2023-10-16T08:20:12.109 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45107.log 2023-10-16T08:20:12.113 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44861.log 2023-10-16T08:20:12.113 INFO:teuthology.orchestra.run.smithi032.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-16T08:20:12.114 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45107.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45107.log.gz 2023-10-16T08:20:12.114 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46976.log 2023-10-16T08:20:12.115 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44861.log.gz 2023-10-16T08:20:12.116 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46976.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46976.log.gz 2023-10-16T08:20:12.116 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51732.log 2023-10-16T08:20:12.117 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2023-10-16T08:20:12.120 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.51732.log: /var/log/ceph/ceph-client.admin.45022.log: 89.9% -- replaced with /var/log/ceph/ceph-client.0.51732.log.gz 2023-10-16T08:20:12.121 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2023-10-16T08:20:12.121 INFO:teuthology.orchestra.run.smithi032.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42731.log 2023-10-16T08:20:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2023-10-16T08:20:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42731.log: 91.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-16T08:20:12.124 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42731.log.gz 2023-10-16T08:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49212.log 2023-10-16T08:20:12.125 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2023-10-16T08:20:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51117.log 2023-10-16T08:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51593.log 2023-10-16T08:20:12.127 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49212.log.gz 2023-10-16T08:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42521.log 2023-10-16T08:20:12.129 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51117.log.gz 2023-10-16T08:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52310.log 2023-10-16T08:20:12.130 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51593.log.gz 2023-10-16T08:20:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42521.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42521.log.gz 2023-10-16T08:20:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49424.log 2023-10-16T08:20:12.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52310.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52310.log.gz 2023-10-16T08:20:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47913.log 2023-10-16T08:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46924.log 2023-10-16T08:20:12.135 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49424.log.gz 2023-10-16T08:20:12.137 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48518.log 2023-10-16T08:20:12.137 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47913.log.gz 2023-10-16T08:20:12.138 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48071.log 2023-10-16T08:20:12.138 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46924.log.gz 2023-10-16T08:20:12.139 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2023-10-16T08:20:12.140 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48518.log.gz 2023-10-16T08:20:12.141 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48071.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.48071.log.gz /var/log/ceph/ceph-client.admin.47693.log 2023-10-16T08:20:12.141 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:12.142 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2023-10-16T08:20:12.143 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45707.log 2023-10-16T08:20:12.144 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42325.log 2023-10-16T08:20:12.144 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2023-10-16T08:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45707.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45707.log.gz 2023-10-16T08:20:12.146 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47993.log 2023-10-16T08:20:12.147 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42325.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42325.log.gz 2023-10-16T08:20:12.148 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2023-10-16T08:20:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43997.log 88.1% 2023-10-16T08:20:12.149 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47993.log.gz 2023-10-16T08:20:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2023-10-16T08:20:12.151 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2023-10-16T08:20:12.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2023-10-16T08:20:12.152 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43997.log.gz 2023-10-16T08:20:12.153 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47560.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2023-10-16T08:20:12.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48701.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2023-10-16T08:20:12.155 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2023-10-16T08:20:12.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45679.log 2023-10-16T08:20:12.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48727.log 2023-10-16T08:20:12.159 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2023-10-16T08:20:12.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52874.log 2023-10-16T08:20:12.160 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45679.log.gz 2023-10-16T08:20:12.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2023-10-16T08:20:12.161 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48727.log.gz 2023-10-16T08:20:12.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52874.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52874.log.gz 2023-10-16T08:20:12.163 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2023-10-16T08:20:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52796.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2023-10-16T08:20:12.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52692.log 2023-10-16T08:20:12.166 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47340.log 2023-10-16T08:20:12.167 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2023-10-16T08:20:12.168 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47797.log 2023-10-16T08:20:12.168 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52692.log.gz 2023-10-16T08:20:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47340.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47340.log.gz 2023-10-16T08:20:12.169 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44633.log 2023-10-16T08:20:12.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46188.log 2023-10-16T08:20:12.171 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47797.log.gz 2023-10-16T08:20:12.172 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2023-10-16T08:20:12.173 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44633.log.gz 2023-10-16T08:20:12.173 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46188.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46188.log.gz 2023-10-16T08:20:12.175 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43783.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2023-10-16T08:20:12.176 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2023-10-16T08:20:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2023-10-16T08:20:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49807.log 2023-10-16T08:20:12.179 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2023-10-16T08:20:12.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50727.log 2023-10-16T08:20:12.180 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2023-10-16T08:20:12.181 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49807.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49807.log.gz 2023-10-16T08:20:12.182 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2023-10-16T08:20:12.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50727.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50727.log.gz 2023-10-16T08:20:12.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2023-10-16T08:20:12.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42351.log 2023-10-16T08:20:12.185 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2023-10-16T08:20:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46794.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2023-10-16T08:20:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2023-10-16T08:20:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42351.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz 2023-10-16T08:20:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47639.log 2023-10-16T08:20:12.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42146.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2023-10-16T08:20:12.189 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45419.log 2023-10-16T08:20:12.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46001.log 2023-10-16T08:20:12.191 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47639.log.gz 2023-10-16T08:20:12.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2023-10-16T08:20:12.192 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45419.log.gz 2023-10-16T08:20:12.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46001.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46001.log.gz 2023-10-16T08:20:12.195 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51892.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz 2023-10-16T08:20:12.196 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44581.log 2023-10-16T08:20:12.197 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44529.log 2023-10-16T08:20:12.198 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45445.log 2023-10-16T08:20:12.199 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44581.log.gz 2023-10-16T08:20:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44529.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44529.log.gz 2023-10-16T08:20:12.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45445.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2023-10-16T08:20:12.202 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46607.log 2023-10-16T08:20:12.203 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52135.log 2023-10-16T08:20:12.205 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2023-10-16T08:20:12.205 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46607.log.gz 2023-10-16T08:20:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52135.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52135.log.gz 2023-10-16T08:20:12.208 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43795.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2023-10-16T08:20:12.209 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2023-10-16T08:20:12.209 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47184.log 2023-10-16T08:20:12.211 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43999.log 2023-10-16T08:20:12.212 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2023-10-16T08:20:12.212 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47184.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47184.log.gz 2023-10-16T08:20:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43999.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43999.log.gz 2023-10-16T08:20:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51410.log 2023-10-16T08:20:12.215 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45237.log 2023-10-16T08:20:12.217 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 2023-10-16T08:20:12.217 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51410.log.gz 2023-10-16T08:20:12.218 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45237.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45237.log.gz 2023-10-16T08:20:12.219 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-16T08:20:12.220 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47771.log 2023-10-16T08:20:12.220 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2023-10-16T08:20:12.221 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47887.log 2023-10-16T08:20:12.223 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43796.log 2023-10-16T08:20:12.223 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47771.log.gz 2023-10-16T08:20:12.224 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42403.log 2023-10-16T08:20:12.224 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47887.log.gz 2023-10-16T08:20:12.225 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49372.log 2023-10-16T08:20:12.226 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43796.log.gz 2023-10-16T08:20:12.227 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2023-10-16T08:20:12.227 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42403.log.gz 2023-10-16T08:20:12.228 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49267.log 2023-10-16T08:20:12.229 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49372.log.gz 2023-10-16T08:20:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45081.log 2023-10-16T08:20:12.230 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2023-10-16T08:20:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.49267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2023-10-16T08:20:12.233 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2023-10-16T08:20:12.233 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45081.log.gz 2023-10-16T08:20:12.233 INFO:teuthology.orchestra.run.smithi032.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.49267.log.gz 2023-10-16T08:20:12.234 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46768.log 87.9% 2023-10-16T08:20:12.234 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz 2023-10-16T08:20:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51920.log 2023-10-16T08:20:12.236 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2023-10-16T08:20:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2023-10-16T08:20:12.237 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46768.log.gz 2023-10-16T08:20:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44607.log 2023-10-16T08:20:12.239 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51920.log.gz 2023-10-16T08:20:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49450.log 2023-10-16T08:20:12.240 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2023-10-16T08:20:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2023-10-16T08:20:12.241 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44607.log.gz 2023-10-16T08:20:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2023-10-16T08:20:12.243 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49450.log.gz 2023-10-16T08:20:12.244 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50275.log 2023-10-16T08:20:12.245 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2023-10-16T08:20:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52458.log 2023-10-16T08:20:12.246 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2023-10-16T08:20:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52484.log 2023-10-16T08:20:12.247 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50275.log.gz 2023-10-16T08:20:12.249 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42459.log 2023-10-16T08:20:12.249 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52458.log.gz 2023-10-16T08:20:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52484.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52484.log.gz /var/log/ceph/ceph-client.admin.48302.log 2023-10-16T08:20:12.250 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42459.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42459.log.gz -5 2023-10-16T08:20:12.252 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45733.log 2023-10-16T08:20:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49531.log 2023-10-16T08:20:12.253 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48302.log.gz 2023-10-16T08:20:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45733.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45733.log.gz 2023-10-16T08:20:12.256 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49531.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49531.log.gz 2023-10-16T08:20:12.442 INFO:teuthology.orchestra.run.smithi186.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-16T08:20:12.474 INFO:teuthology.orchestra.run.smithi186.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-16T08:20:12.514 INFO:teuthology.orchestra.run.smithi186.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-16T08:20:12.559 INFO:teuthology.orchestra.run.smithi186.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-16T08:20:12.561 INFO:teuthology.orchestra.run.smithi186.stderr: 2023-10-16T08:20:12.561 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m0.896s 2023-10-16T08:20:12.562 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m3.347s 2023-10-16T08:20:12.562 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m0.290s 2023-10-16T08:20:12.604 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-16T08:20:12.606 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-16T08:20:12.749 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-16T08:20:12.795 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-16T08:20:12.797 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-10-16T08:20:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m1.151s 2023-10-16T08:20:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m4.146s 2023-10-16T08:20:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m1.098s 2023-10-16T08:20:12.799 INFO:tasks.ceph:Archiving logs... 2023-10-16T08:20:12.800 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/remote/smithi032/log 2023-10-16T08:20:12.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T08:20:13.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-16T08:20:13.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-10-16T08:20:13.312 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-10-16T08:20:13.601 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/remote/smithi186/log 2023-10-16T08:20:13.602 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T08:20:14.027 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-16T08:20:14.036 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-16T08:20:14.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T08:20:14.041 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T08:20:14.125 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-16T08:20:14.134 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-16T08:20:14.135 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:20:14.139 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:20:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:20:14.152 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2023-10-16T08:20:14.153 INFO:teuthology.orchestra.run.smithi032.stdout:*hv01.front.sepi 44.4.53.2 2 u 2 64 377 0.073 -1.411 0.647 2023-10-16T08:20:14.153 INFO:teuthology.orchestra.run.smithi032.stdout:+hv02.front.sepi 63.231.80.2 3 u 65 64 177 0.084 +1.130 0.642 2023-10-16T08:20:14.153 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 74.6.168.72 3 u 67 64 177 0.077 +0.660 0.589 2023-10-16T08:20:14.154 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:20:14.155 INFO:teuthology.orchestra.run.smithi186.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:20:14.155 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================== 2023-10-16T08:20:14.156 INFO:teuthology.orchestra.run.smithi186.stdout:*hv01.front.sepi 44.4.53.2 2 u 1 64 377 0.070 -1.616 0.999 2023-10-16T08:20:14.156 INFO:teuthology.orchestra.run.smithi186.stdout:+hv02.front.sepi 63.231.80.2 3 u 66 64 177 0.243 +0.353 1.074 2023-10-16T08:20:14.156 INFO:teuthology.orchestra.run.smithi186.stdout:+hv03.front.sepi 74.6.168.72 3 u 64 64 177 0.128 -0.005 1.020 2023-10-16T08:20:14.156 INFO:teuthology.orchestra.run.smithi186.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:20:14.157 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-16T08:20:14.167 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-16T08:20:14.168 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-16T08:20:14.182 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-16T08:20:14.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-16T08:20:14.210 INFO:teuthology.task.internal:Duration was 990.252913 seconds 2023-10-16T08:20:14.211 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-16T08:20:14.226 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-16T08:20:14.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T08:20:14.230 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T08:20:14.340 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-16T08:20:14.341 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-10-16T08:20:14.341 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T08:20:14.364 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T08:20:14.365 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T08:20:14.387 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-16T08:20:14.388 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T08:20:14.415 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T08:20:14.463 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-16T08:20:14.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T08:20:14.496 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T08:20:14.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-16T08:20:14.596 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-16T08:20:14.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T08:20:14.618 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T08:20:14.639 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-16T08:20:14.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:20:14.663 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:20:14.678 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-10-16T08:20:14.700 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-10-16T08:20:14.719 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:20:14.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:20:14.747 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:20:14.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:20:14.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-16T08:20:14.781 INFO:teuthology.task.internal:Transferring archived files... 2023-10-16T08:20:14.781 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/remote/smithi032 2023-10-16T08:20:14.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T08:20:14.816 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915/remote/smithi186 2023-10-16T08:20:14.817 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T08:20:14.841 INFO:teuthology.task.internal:Removing archive directory... 2023-10-16T08:20:14.842 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T08:20:14.867 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T08:20:14.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-16T08:20:14.908 INFO:teuthology.task.internal:Not uploading archives. 2023-10-16T08:20:14.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-16T08:20:14.933 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-16T08:20:14.933 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T08:20:14.937 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T08:20:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: 657968 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 08:20 /home/ubuntu/cephtest 2023-10-16T08:20:14.951 INFO:teuthology.orchestra.run.smithi186.stdout: 657961 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 08:20 /home/ubuntu/cephtest 2023-10-16T08:20:14.953 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-16T08:20:14.964 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-16T08:20:14.987 INFO:teuthology.nuke:Checking targets against current locks 2023-10-16T08:20:15.010 DEBUG:teuthology.nuke:shortname: smithi032 2023-10-16T08:20:15.011 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T08:20:15.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 07:54:44.943355', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqvVuFrg37eMep8ddVjkwbT5zCcErGvSYFtwEG/mXOCYAfK/iBsoIRy/HyXeG6IN0IuqYZf+79xZY0ZbxuRTuc='} 2023-10-16T08:20:15.037 DEBUG:teuthology.nuke:shortname: smithi186 2023-10-16T08:20:15.038 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T08:20:15.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_06:54:33-fs-rishabh-2023oct13-testing-default-smithi/7428915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 07:54:44.945321', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8sPORMgXFVHvX5dfFzOhiDr4AyqKoQe4VOSoiyeAbGE2pFEXowpG0s50XRUwhLwjVPkXWoBj9ftaoMz8BBBHY='} 2023-10-16T08:20:15.077 INFO:teuthology.orchestra.console.smithi032:Power off 2023-10-16T08:20:15.078 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T08:20:15.101 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-10-16T08:20:15.101 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:15.121 INFO:teuthology.orchestra.console.smithi186:Power off 2023-10-16T08:20:15.121 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T08:20:15.131 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-10-16T08:20:15.143 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-10-16T08:20:15.144 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:15.169 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-16T08:20:19.133 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:19.171 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:19.244 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-10-16T08:20:19.283 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-16T08:20:23.248 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:23.284 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:23.360 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-10-16T08:20:23.393 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-10-16T08:20:23.493 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-10-16T08:20:27.362 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:20:27.472 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-10-16T08:20:27.573 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-10-16T08:20:27.678 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 990.2529134750366 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 mon.b-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0a17552bbd354a94856fe88fb057eb4a status: fail success: false 2023-10-16T08:20:27.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T08:20:27.753 INFO:teuthology.run:FAIL